Visual Shortcuts | ODeV framework

A bit of context

Keyboard is one of the oldest input device. Their history start around 1970, and one of the first example was the Altair Computer with its keyboard (implemented as a set of key switches).

Courtesy of the Computer History Museum

Since then the technology to implement a modern keyboard is improved, they come in many different shapes, colors and sizes, they are wireless or connected via cable. The development of smartphones and tablets has brought us the virtual keyboard, the concept of keyboard cover, and some of them have also a trackpad like the laptops, but the basic concept is still the same. And this is because a keyboard just works.

So, is there any more room to improve the keyboard concept? Is there anything else to create something new?

The Idea

A feature I like when I use a physical keyboard are the shortcuts. That is a powerful way to send commands to the active application or the OS. The main problem is that one shortcuts can involve also many keys like CTRL + ALT + SHIFT + A, and every application can have many shortcuts. So they are not so easy to find and difficult to remember.

Macintosh menus with keyboard shortcuts specified in the right column
(source wikipedia)

In this scenario the idea is to put a display in the keyboard, and this is the initial requirements list for a new feature that we call Visual Shortcuts:

To display the most common and useful shortcuts.
The content (the displayed shortcuts) changes according to the active application.
The user can customize the shortcuts for each supported application.
The user an turn off / on the display.

A Visual Shortcut is one smart virtual key for a shortcut, easy to find and available when you need it.

Visual Shortcuts

There are many open points to implement the Visual Shortcuts, and many questions are to be addressed, starting for example, from the hardware design. Where to put the touch display?
The ideal place could be above the trackpad or it could be also integrated with the trackpad.

But the purpose of this project is to focus on the firmware. I want to test and analyze the utility of the Visual Shortcuts for the user, to define an API and a protocol in order to allow application running in the host processor to support the Visual Shortcuts.

Main Components

STM32L4R9I-EVAL board. I am not an hardware engineer, so I have to select what is available, and this is a good board to prototype the idea presented in this page.
A keyboard or keypad with an accessible key matrix to be connected to STM32.
ODeV framework. Of course it is the base for the firmware.
TouchGFX, a powerful UI framework for STM32. Moreover I have already addressed the integration of TouchGFX with ODeV in this blog post.
ST USB Library. It is part of STM32Cube HAL. I want to connect the prototype to the host PC via USB.
Atollic TrueSTUDIO. The preferred IDE.

Fig.01 displays an high level block diagram of the prototype.

Fig.1 - Block diagram

The Project

The source code of the project is available on the GIT repository.

Conclusion and nexts steps

In this page I introduced the idea of Visual Shortcuts. The prototype is a work in progress, and I will update this page during the development. Another place for the documentation will be the API documentation, as usual for each ODeV based application.