A couple of months ago (more like a year, but I kinda forgot about it) I managed to grab a Stream Deck on Amazon for around half the original price. A nice deal. The thing is, I’m not a streamer. I wasn’t thinking of using the “Stream Deck” to “Stream”, I was just interested in the “Deck” part… I guess.
Elgato (the creators of the Stream Deck) know that people like me exist, and they made sure the Stream Deck can be used with multiple software and workflows, completely unrelated to streaming, by releasing plugins you can install from the Stream Deck’s controller software. And, if the software or use case you have in mind that is not covered by them, there is a full range of available SDK’s for you to play around with.
With this information, I decided to purchase the Stream Deck and use it as a shortcut / Unity Menu extension for my work. I though that by then there should definitely be some plugin around the Internet to allow communication between the Stream Deck and Unity (a little spoiler, there was not).
Apparently, the available SDK’s work with Javascript, Objective-C and C++, but there is no official implementation for C#. Now a days there are multiple unofficial wrappers of the SDK written in C#, but all of them are Windows only, and all of them require me to do stuff outside of Unity, which I wanted to avoid in this case.
Defeated, I tried to use the basic “shortcut” functionality and see if I could get some value out of this thing.
Why basic shortcuts just don’t work
My work projects have a lot of tools. And those tools come with a lot of MenuItems to trigger them. Clean Player Prefs, download localizations, run tests, you name it. The main propose of the Stream Deck was to allow me to trigger this buried MenuItems directly, being able to group them together in folders, and even create small “scripts” that would trigger regularly-used combinations of shortcuts or MenuItems, all at once.
The Stream Deck has the basic functionality to trigger key combinations, or shortcuts, as an action. This will essentially send a bunch of key presses like you just pressed them in your keyboard, pretty simple stuff.
This approach worked… for the first 5 actions or so. I could trigger any MenuItem I wanted without issues, as long as it had a shortcut pre-setup on it.
And that’s the first issue with this approach.
If the MenuItem you want to trigger does not have a shortcut assigned to it, you won’t be able to trigger it. And, if you happen to be using some code that you shouldn’t change, or can’t change, the ability to add a shortcut to it is not possible (I know about the Shortcut Manager in 2019.3, but the point still stands).
And as I said, that’s only one issue.
Since this type of actions are just shortcuts, the Stream Deck can’t possibly understand the current context of the Editor, or where is your focus. Although you can assign specific layouts for specific software (so for example, it will only work if your focus is in Unity, and not, let’s say, Chrome) that does nothing to know if you are in edit or play mode, if a GameObject is selected, if another window inside the Editor has the focus, etc. This seemingly small issue became a blocker in my case pretty quickly.
Deep linking everything
I couldn’t bear the idea that I wasted so much money in a paperweight with a screen on it, so I tried to investigate how to make this thing recognizable enough in Unity to be at least “usable”.
One of the basic actions that the Stream Deck can assign an action to is opening a website. Well, to be more specific it allows to open any url using the default browser. That’s the important part.
In case you didn’t know, URL’s aren’t just pointing to some IP’s of some server, but they are widely used to reference multiple protocols, like ftp: for file transfers, or mailto: for opening your default email software. You probably encountered this last when clicking on a “Contact” link on a website, where it basically triggered your default email software to just show up with a new email ready for you to send. It’s pretty annoying to be honest, but pretty useful. In this case your system knows that any URL starting with mailto: means “write a new email”, so it opens your email software.
This is called “deep linking”. It’s widely used in mobile apps to redirect requests to installed apps instead of opening the browser with a probably inferior mobile site. Like any other URL or URI, we can attach any query params or payload to be handled by the target app.
This is exactly what we need!
The issue is, to create deep links that software will recognize, they must be added as part of the core logic of the software before compiling it, which means that unless I want to re-compile the Unity Editor, we need to use any of the available deep links that Unity already listens for.
Asset Store to the rescue
Thankfully, the Unity Editor has one deep link setup for us to hijack. If you ever downloaded any asset from the Asset Store, you have probably used the handy “Open in Unity” button to trigger a download in Unity without having to copy paste any files. This button is essentially a deep link!
With this discovery, we can start to reverse engineer how the editor recognizes this action as a “download this asset” callback.
We will start on the asset store itself. If we inspect the button to open Unity we see it contains some data attributes, but sadly there is no URL. Instead it seems it uses Javascript to trigger the click event internally. Well, thankfully we can spy the network activity to check exactly what the website is trying to call. We just need to click on the button while checking the network activity…
And there it is! Apparently it tried to invoke com.unity3d.kharma:content/32647 (among other identifiers). We can also see that the data after content it’s just the internal asset ID, which is used in the Asset Store page.
With this information, we now know that com.unity3d.kharma: is the deep link the Unity Editor will listen to. But this is only one part of the issue, as trying to use this URL with any random data, will open the Asset Store, and then fail to show anything, as it doesn’t understand what just happened.
We need to find a way to steal the full URL from the Asset Store Editor window and use it for our own scripts.
Ugly Reflection
I’ll go over some code to show the train of thought a bit clearer, for a technical explanation and source code, check the README in the repository.
If you ever tried to reflect any variables or methods out of a class that’s out of reach, you must know the first step is assuming stuff. So, let’s assume that if the Asset Store URL contains the ID of which asset to download / show, this must be parsed somewhere inside the Asset Store Editor window. So, we first need to check if the Asset Store window is open or not:
private bool _isWindowOpen = false;
static HookInterceptor() {
// Subscribe OnUpdate() to the Editor update in the constructor
EditorApplication.update += OnUpdate;
}
private static void OnUpdate() {
// Get all loaded objects that derivate for EditorWindow (therefore, all editor windows)
var windows = Resources.FindObjectsOfTypeAll<EditorWindow>();
if (windows == null || windows.Length <= 0) return;
foreach (var window in windows) {
// Check if the window title is "Asset Store". Easier than reflecting the type
if (window.titleContent.text != "Asset Store") continue;
_isWindowOpen = true;
return;
}
// No Asset Store found, mark as closed
_isWindowOpen = false;
}
We now had a bool that will return true if the Asset Store Editor window is opened, great! Of course, this needs some checks to differentiate manually opened from “deep linked” opened windows, among other small improvements. This is supported in the repository, but I won’t go over it here, is pretty boring.
Now that we know if the window is open, we need to steal the URL, which we assume is stored… somewhere.
This took so much more time to find that I would like to admit. Essentially, I added a breakpoint in the previous code, and went backwards from there. In the end, we can get the URL with the following code:
// AssetStoreContext contains all the data from the Asset Store Editor window
var type = _assembly.GetType("UnityEditor.AssetStoreContext");
// GetInstance() will return the current instance
var instance = type.GetMethod("GetInstance").Invoke(null, null);
// GetInitialOpenURL() will return the URL used to open this window, if any
string url = (string)instance.GetType().GetMethod("GetInitialOpenURL").Invoke(instance, null);
And there it is. url contains the URL used to open the Asset Store window. In hindsight it looks a lot simpler than expected, but finding this specific methods wasn’t easy… And any future updates will break this, but that’s the price you pay for using Reflection.
Stream Deck configuration
From here forward, I decided to add a bunch of features and utilities to make this ugly reflections easy and simple to use for the end user. Again, the technical documentation about all the supported features, attributes and callbacks is in the repository’s README.
Trying to make the Stream Deck I ended up creating a custom deep link handler with support with payloads (I like long names), which basically allows any software or hardware that has the capability of invoking URI’s to the system to invoke methods inside a Unity Editor.
But let’s not lose focus, the point of all of this is to make Stream Deck call the Unity Editor, so let’s configure that now.
With the repository’s code in the project, we can make use of the very easy Attributes to define a method or property that should listen to events:
private HookExample() {
// Attach this class attributes to the main logic
HookAttributesParser.Add(this);
}
[HookMethod(new[] {"test"})]
public void Testing() {
Debug.Log("Works!");
}
And finally, call the URI com.unity3d.kharma:hook/test from Stream Deck as an action.
Clicking on it should create a log in the Unity console. Easy, huh?
Conclusion
I wanted to make this entire thing open source because is based on assumptions, which means that is a matter of time until Unity changes their internal API, or get’s rid of the Asset Store entirely (as it seems likely with the new Package Manager), and this entire hack will stop working. Hopefully I can keep it in working order for as long as possible, or until somebody can produce a good enough plugin for Stream Deck that can deal with this communication.
Until that happens, feel free to use all of this for any variant of Stream Deck (including the weird mobile app), or anything that triggers URI’s, actually.