Add simple utilitity to control the touchbar.

[roadrunner2]

Awesome work on the capture! 👍 From that I created a small utility to manually control the touchbar, mainly to experiment a bit. Doing so I found a total of 4 commands: show escape-only, show function-keys, show feature-keys, and disable-touchbar. As you noted, showing the feature keys still generates KEY_FN* events; but by showing the function keys you get all of them (i.e. FN2 and FN3 too).

In order to try and reverse engineer more functionality, I brought up MacOS in a VBox VM and passed the iBridge usb device through too, like you did, but haven't been able to get it actually recognize the touchbar (it does show the iBridge usb device, just like when booting MacOS directly, but the touchbar stays dark). So more work needed here.

I'm going to do a preliminary integration into the applespi driver in the next few days and see if cb22 is interested in accepting that.

[Dunedan]

Wow, this is really awesome. 👍

Although I don't know if it makes sense to include it into applespi, as the Touch Bar a completely separate topic from the SPI-input stuff. Maybe a separate driver would make more sense, but I guess you already thought about that.

Regarding the iBridge device in a VM, ensure you fake DMI values of the VM so they state that it's e.g. a MacBookPro13,2. I had some trouble getting the Bootcamp drivers to work in the Windows VM as well, until I properly set those values. With VirtualBox DmiSystemProduct and DmiBoardVersion seemed to be the key there.

[Dunedan]

Oh btw: In the meanwhile I also figured out how the commands to dim, switch off and switch on the Touch Bar look like. I'll add them later to the repo.

[roadrunner2]

Although I don't know if it makes sense to include it into applespi, as the Touch Bar a completely separate topic from the SPI-input stuff. Maybe a separate driver would make more sense, but I guess you already thought about that.

I did, but on further thought it does make more sense as a separate driver - will basically need to attach an input handler to the touchbar input device and filter the keys so as to be able to turn the FN keys into proper feature-keys when in feature mode.

Regarding the iBridge device in a VM, ensure you fake DMI values of the VM so they state that it's e.g. a MacBookPro13,2. I had some trouble getting the Bootcamp drivers to work in the Windows VM as well, until I properly set those values. With VirtualBox DmiSystemProduct and DmiBoardVersion seemed to be the key there.

Thanks for the tip; but I faked everything in dmi (that can be found in /sys/devices/virtual/dmi/id/) and it still didn't work.

Oh btw: In the meanwhile I also figured out how the commands to dim, switch off and switch on the Touch Bar look like. I'll add them later to the repo.

The suspense is killing me! Well, I found the off and on (see the usage in touchbar.c), but not the dimming. Unless that's the other command? (there were two commands in the trace you provided, but the first did not appear to do anything, so I've ignored it). Look forward to your update.

[Dunedan]

Here you go: https://github.com/Dunedan/mbp-2016-linux/blob/master/touchbar-dimming.pcapng
I added packet comments to the relevant packets to make it easier to understand what they're used for.

[Dunedan]

Oh, btw: I got macOS in a VM working with the Touch Bar. Worked out-of-the-box after installing the most recent macOS update. Full packet capture of everything macOS does with the iBridge during a regular boot is available here: https://github.com/Dunedan/mbp-2016-linux/blob/master/touchbar-interaction-macos.pcapng.xz

[roadrunner2]

Thanks for those traces! Grumble, I wonder why the VM is working for you but not for me (and yes, I updated to the latest version of MacOS). Oh well.

The traces are interesting: so there's a simplified interface on usb-interface 2, and the full monty is on usb-interface 3. The latter will need some more investigation.

For now I've implemented a basic driver based on the simplified interface - it's currently available from my clone of the keyboard driver at https://github.com/roadrunner2/macbook12-spi-driver.git in the touchbar-driver-hid-driver branch. I updated the instructions on my gist, but basically pull from my repo, switch to the above branch, build, and add the new appletb driver in the same places you did the applespi driver.

[Dunedan]

Awesome! Just works. It even got a configurable switch-off timeout. That's a feature even macOS doesn't have! 😄

Am I right that there is currently no way to toggle the default behavior of the fn-key like the hid_apple module offers with its fnmode parameter? That's the only feature I currently miss.

Aside from that: Any plans to upstream that driver already so we can get support for this basic functionality of the Touch Bar out of the box in Linux distributions?

[roadrunner2]

Toggling the default fn-key behaviour is on my todo list - next weekend...

As to upstreaming, I'd absolutely like to see this in the regular kernel. However, it currently does have some implicit dependencies on the applespi driver (it looks for an SPI keyboard and touchpad so that it won't trigger on events or fn-keys from external keyboards and mice) which might be problematic. But I'm sure things can be worked out. It would be really nice to get the applespi driver upstreamed too, though (though I presume the flaky touchpad init needs to get fixed first, and the caps-lock led would be nice too).

As a side note on the switch-off timeout: I compared the power usage (well, current drain on the battery, actually) when the touchbar was on vs off (with usb suspend enabled), and the difference is only about 200mW ±30% or so. So it's fairly little (about 1% at current usage, which I hope will come down eventually), and hence reducing the timeout buys you almost nothing.

[Dunedan]

Toggling the default fn-key behaviour is on my todo list - next weekend...

👍 Excellent. I'm already looking forward to next Monday 😄

As to upstreaming, I'd absolutely like to see this in the regular kernel. However, it currently does have some implicit dependencies on the applespi driver (it looks for an SPI keyboard and touchpad so that it won't trigger on events or fn-keys from external keyboards and mice) which might be problematic.

Good point.

It would be really nice to get the applespi driver upstreamed too, though (though I presume the flaky touchpad init needs to get fixed first, and the caps-lock led would be nice too)

I totally agree, but right now it looks to me that your Touch Bar driver is already in a better shape than applespi. I mean there is still a lot to do if we're looking at the advanced functionality of the Touch Bar, but the basic functionality is already working pretty well.
If you have ideas how to solve the remaining issues with applespi and there is something I can help with, just drop me a note.

As a side note on the switch-off timeout: I compared the power usage (well, current drain on the battery, actually) when the touchbar was on vs off (with usb suspend enabled), and the difference is only about 200mW ±30% or so. So it's fairly little (about 1% at current usage, which I hope will come down eventually), and hence reducing the timeout buys you almost nothing.

Well, my intention was not to decrease the timeout, but to increase it significantly. I prefer my keys not to vanish suddenly, so I immediately increased the timeout to a few hours. 😉

[roadrunner2]

If you have ideas how to solve the remaining issues with applespi and there is something I can help with, just drop me a note.

Basically we need traces. For the caps-lock led cb22/macbook12-spi-driver#3 said he had some, but if he can't find them we'll need new ones. For the initialization it looks like we need some from a MacBookPro13,*, which can probably be acquired by running Windows on VBox with PCI passthrough for the SPI PCI device (the applespi driver probably needs to be unloaded and an external keyboard used so that there's no conflicts, though).

Well, my intention was not to decrease the timeout, but to increase it significantly. I prefer my keys not to vanish suddenly, so I immediately increased the timeout to a few hours. 😉

One ugly thing with using the simple mode is that off really means off, and hence you can't wake the touchbar by tapping on it (only keyboard or touchpad will wake it); so wanting to increase the timeout makes a lot of sense. I might change the default timeout to something like 5 min.

[Dunedan]

For the initialization it looks like we need some from a MacBookPro13,*, which can probably be acquired by running Windows on VBox with PCI passthrough for the SPI PCI device

I'll see if I can get that working.

One ugly thing with using the simple mode is that off really means off, and hence you can't wake the touchbar by tapping on it

Have you had a look ad my dimming trace (https://github.com/Dunedan/mbp-2016-linux/blob/master/touchbar-dimming.pcapng)? I think the commands in there might be the key for having it switched off, but still active for touch input.

[roadrunner2]

Toggling the default fn-key behaviour is on my todo list - next weekend...

👍 Excellent. I'm already looking forward to next Monday 😄

I just pushed an update with this functionality. Note that the both the fn-mode and the idle-timeout are now device attributes (typically found under /sys/bus/hid/devices/0003:05AC:8600:0001/); but you can set defaults via module params at boot time.

One ugly thing with using the simple mode is that off really means off, and hence you can't wake the touchbar by tapping on it

Have you had a look ad my dimming trace (https://github.com/Dunedan/mbp-2016-linux/blob/master/touchbar-dimming.pcapng)? I think the commands in there might be the key for having it switched off, but still active for touch input.

Yes, I did, but wasn't sure that mixing accesses to the two interfaces would work. However, some subsequent quick testing shows that it does indeed work, and the "off" command on interface 3 is really just a display off, i.e. full functionality remains, including mode switching (you just can't see the keys). I need to play with it a bit more and see how much power is saved by switching off the display - otherwise I'm not sure what the point of doing so is.

[Dunedan]

I had no success to far getting the keyboard working via SPI in a VM, but I'll keep trying.

Regarding the device attributes: I feels kind of odd that the module params are now prefixed with default_. Afaik that's usually not the case for other kernel modules. I suggest to call them fnmode (to use the same wording as hid-apple) and idle_timeout.

Yes, I did, but wasn't sure that mixing accesses to the two interfaces would work. However, some subsequent quick testing shows that it does indeed work, and the "off" command on interface 3 is really just a display off, i.e. full functionality remains, including mode switching (you just can't see the keys).

Why do you use interface 3? If I'm not mistaken my dimming trace shows the use of interface 0 for the dimming commands. The only time interface 3 is used in this trace is to pass the interrupts from the device to the host.

[roadrunner2]

Regarding the device attributes: I feels kind of odd that the module params are now prefixed with default_. Afaik that's usually not the case for other kernel modules. I suggest to call them fnmode (to use the same wording as hid-apple) and idle_timeout.

I've been going back and forth in my mind a bit on this. The thing is that the params are now actually properly attached to the hid device, and can be set/read via /sys/bus/hid/devices/0003:05AC:8600:0001/{fn_mode,idle_timeout} (and changes on those properly take effect immediately). But in order to be able to provide a different default at boot time, I decided to still provide module parameters, though they are read-only now (i.e. can only be set when loading the module, generally via /etc/modprobe) and, as their name indicates, they provide the default values to use when the actual device is set up.

However, I do realize this is a bit different than other modules - they generally provide either module params or device attributes, but not both.

Why do you use interface 3? If I'm not mistaken my dimming trace shows the use of interface 0 for the dimming commands. The only time interface 3 is used in this trace is to pass the interrupts from the device to the host.

I think you may be confusing "Endpoint" with "Interface" (and yes, the USB terminology takes a bit of getting used to...) If you look at the trace you'll see the bmRequestType is of type Class with recipient Interface - this means that wIndex specifies the interface the command is destined for; so it's destined for endpoint 0, but interface 3.

[roadrunner2]

FYI, I just pushed some updates to my fork and now it will dim the touchbar as well as allowing touches on the touchbar to wake it up.

[mhkabir]

@roadrunner2 Nice! Will test :) Do you have any ideas about the flaky SPI behaviour for the touchpad? It's probably the most annoying issue which I'm no closer to solving yet.

[roadrunner2]

@mhkabir I presume you mean the initialization? See the discussion on cb22/macbook12-spi-driver#6 - need good traces.

[Dunedan]

@roadrunner2 Awesome. Dimming as well as waking up via touch works. 👍

There are a few observations I made:

• The commit message states:

Dimming is now performed after 2/3 of the idle-timeout or 30 seconds,
whichever is smaller.

That'd translate to min(idle timeout * 2/3, 30 seconds). Instead it's idle timeout - min(30 seconds, idle timeout / 3) which results in dimming the Touch Bar maximum 30 seconds before the idle timeout is reached. That's pretty odd for large idle timeouts, because the dimmed state will only take up a tiny fraction of the active time of the Touch Bar. In general it feels a bit too static for different lengths of idle timeouts.

In addition to that I'd love to see the ability to configure APPLETB_MAX_DIM_TIME in the same way as it's already possible with the idle timeout.

• Maybe somehow related to the previous point, but really just a minor thing: If no interaction with the input devices happens after loading the kernel module, the Touch Bar directly switches off without dimming at the time when the dimming should happen.

• Waking up the Touch Bar by touching it doesn't wake up a shut-off main display as regular key presses or interactions with the Touchpad do (at least until you press one of the keys of the woken up Touch Bar).

[roadrunner2]

@Dunedan Thanks for reviewing.

There are a few observations I made:

The commit message states:

Dimming is now performed after 2/3 of the idle-timeout or 30 seconds, whichever is smaller.

That'd translate to min(idle timeout * 2/3, 30 seconds). Instead it's idle timeout - min(30 seconds, idle timeout / 3) which results in dimming the Touch Bar maximum 30 seconds before the idle timeout is reached. That's pretty odd for large idle timeouts, because the dimmed state will only take up a tiny fraction of the active time of the Touch Bar. In general it feels a bit too static for different lengths of idle timeouts.

You're absolutely right, got the message confused.

In addition to that I'd love to see the ability to configure APPLETB_MAX_DIM_TIME in the same way as it's already possible with the idle timeout.

I was trying to avoid too many knobs and all the associated testing... On the whole I'm actually not sure what the dimming really provides, except functioning as a warning that the display is about to be turned off, and as such I think it makes sense that it's never very long, even for long idle timeouts (similar to screens that dim a bit or do a slow transition when going blank - that time tend to be fixed no matter how long the inactivity timeout). But I'm open to discussion.

Maybe somehow related to the previous point, but really just a minor thing: If no interaction with the input devices happens after loading the kernel module, the Touch Bar directly switches off without dimming at the time when the dimming should happen.

Ah, good catch - fixed (will push a little later). Only happened if you don't touch the keyboard, touchpad, or touchbar after loading the module and before the dim timeout expires, which is why I missed this.

Waking up the Touch Bar by touching it doesn't wake up a shut-off main display as regular key presses or interactions with the Touchpad do (at least until you press one of the keys of the woken up Touch Bar).

Hmm, hadn't thought about that, but you're right, that makes sense - I guess I need to generate a dummy input event when waking (the actual key event is supressed, because you're essentially blindly touching the touchbar and hence it's unclear what key you happened to hit). Ok, will be part of next push.

[Dunedan]

I was trying to avoid too many knobs and all the associated testing

I'm a big fan of sane defaults, but the ability to configure behavior for advanced use. I don't feel making this timeout configurable adds too much complexity, but I guess I can live without it too.

What'd be really nice would be to use the dimmed setting for operation in darker environments, but that has of course a working ambient light sensor as requirement.

[roadrunner2]

@Dunedan FYI, I pushed updates to my clone with fixes for the two issues you mentioned, as well as various applespi fixes (initialization + caps-lock-led support).

[Dunedan]

I already saw you PR (cb22/macbook12-spi-driver#14). 👍
Will try it out later today or tomorrow morning.

[mhkabir]

@roadrunner2 Just tried it ^_^. Works perfect, thanks!

[Dunedan]

@roadrunner2 I noticed a new bug in appletb: Frequently pressing the fn-button can get the Touch Bar into a state were it shows the wrong keys until the fn-button is touched once more again. It's also possible through this frequent pressing to get the Touch Bar into a state where it doesn't react to fn-key presses for several seconds.
I was able to get it reproducible into these states, although reproducing is quite tricky and might require pressing the buttons quite a lot.

P.S.: If you'd enable issues on https://github.com/roadrunner2/macbook12-spi-driver I could directly open issues there instead relying on this old PR.

[roadrunner2]

@Dunedan Oops, didn't realize issues were disabled - fixed now. Thanks for pointing it out. And yes, opening an issue there would be a better place to discuss this, I think.