Going places I shouldn't with JNA
》Why C when you can CLJ?
Someone close to me says when looking at dangerous animals: "if not friend, then why friend shaped?" I feel like that about C. While C isn’t exactly dangerous, the slow write-compile-test cycle when when working on visual projects is a real drag.
Which is a shame because I’ve done a number of visual projects in Clojure over the years and have always wanted to incorporate them into XScreenSaver. I even wrote a hack for XScreenSaver once in C. Though it worked in the end, getting there was a slog. And in the end, JWZ wouldn’t accept it because it didn’t compile on his Mac.
[exasperated emoji here]The workings of an XScreenSaver hack aren’t complicated - the main
xscreensaver daemon creates an empty window and then launches another process
(the "hack") to draw remotely on the window. That’s easy to do in X11 by
passing along the window id from the daemon to the hack.
Surely that can be done with Java/Clojure, right?
Well read on and see exactly how wrong I was.
》The Early Attempt
My thinking was: if I can just get AWT to use the window ID when creating a new window, that should do it!
The first idea was to dig through the OpenJDK code and find where XLib’s
XCreateWindow function is called. That will reveal where AWT gets the
window ID of the newly created window, and with luck where it is kept.
XCreateWindow turns up a few times, but this line in
XBaseWindow.java
looks like the right one.
/**
* Creates window using parameters {@code params}
* If params contain flag DELAYED doesn't do anything.
* Note: Descendants can call this method to create the window
* at the time different to instance construction.
*/
protected final void init(XCreateWindowParams params) {That didn’t help much in trying to figure out how to change the window ID.
The next idea was to create a window in Java and trace through to where
XBaseWindow shows up. If my understanding is correct, most AWT objects have a
"peer" native object. So I tried creating an AWT Frame and getting the
ComponentPeer from it. And then used reflection to see the window field
which holds the window ID. If that value could be changed to the foreign
window’s ID, then the rest of the AWT calls might just draw on it.
You can see my attempt to used reflection to change the window ID here in the
AWTForeignXWindow repository.
Maybe someone more clever than me can make this work. Even after dozens of
add-opens and add-exports passed to the JDK to get access to sun.awt and
java.awt classes, Java just would not allow me to change that value.
I could not find a way to inject an external window ID.
Another option appeared briefly promising: the vestigial objects from the days of Java applets. Those pretty much worked like the XScreenSaver hacks do - the browser would create an embedded X11 window and pass the window ID to Java.
EmbeddedFrame.java looks like the right place. Even the comments are fire:
* Within a C-based application, an EmbeddedFrame contains a window
* handle which was created by the application, which serves as the
* top-level Java window. EmbeddedFrames created for this purpose
* are passed-in a handle of an existing window created by the
* application. The window handle should be of the appropriate
* native type for a specific platform, as stored in the pData field
* of the ComponentPeer.Luck was not with me. Though the code looked correct and did not produce any errors, nothing happened on the target window.
》A Renewed Effort, Then Despair
A completely random post about Clojure and JNA put me back into the mood to make this work. Notably the JNA platform has X11 support baked in! Just look at all this X11 goodness:
VisualID XID Atom AtomByReference Colormap Font Cursor
KeySym Drawable Window WindowByReference Pixmap Display
Visual Screen GC XImage Aspect XWindowAttributes
VisualInfo ... ... ...Yes! Just the sort of functions I was hoping to see exposed. Straight up XLib calls.
Even though this doesn’t solve the problem of using AWT, it opens the possibility of using plain X11 drawing functions. In the spirit of adventure, I gave it a shot.
Let’s dig in to some Clojure! First up, this is the import list. JNA’s constructs for memory pointers and longs, followed by X11 constructs.
(:import
[com.sun.jna Memory NativeLong]
[com.sun.jna.platform.unix X11]
[com.sun.jna.platform.unix X11$XGCValues X11$Window X11$XWindowAttributes]
[java.awt.image BufferedImage]
[java.awt Point])》》A Connection Is Made
The code below is one chamber of the heart of the project. It establishes the X11 connection and "finds" the foreign window to draw to. The other chamber I’ll explain later - it handles sending images to the X server.
The Xlib Manual at tronche.com is useful if you want to understand more.
(def x (X11/INSTANCE))
(def dpy (.XOpenDisplay x (System/getenv "DISPLAY")))
(def xgcv (new X11$XGCValues))
(def win (new X11$Window window-id ))
(def xgc (.XCreateGC x dpy win (new NativeLong 0) xgcv))
(def xwa (new X11$XWindowAttributes))
(.XGetWindowAttributes x dpy win xwa)
(def width (.width xwa))
(def height (.height xwa))
(def depth (.depth xwa))The
$DISPLAYvariable is read from the environment and opened.An empty
XGCValues— that’s X Graphics Context Values — is created.A
Windowis created from the window ID passed in to the program.A graphics context created for the foreign window.
XWindowAttributesare created and populated.The width/height/depth are read from those attributes.
Then a buffer is created to store an image.
With that, there is enough to start drawing! I used the standard xwinfino
command line tool to target a terminal window and get that window’s ID, then
ran through the code above.
The JNA libs provide functions like XDrawPoint and XFillRectangle to do the
drawing. I played around with these and reached a point that I could draw some
rectangles on the foreign window!
But I also saw some fatal problems with this approach:
The functions for setting background/foreground color are undocumented.
Drawing functions are not antialiased.
JNA doesn’t expose
XDrawLine!
That last one alone is enough to kill the project. I left it alone for a few weeks.
》Inspiration Strikes
Then one day came a thought: could the Java BufferedImage in memory have the
same format that XPutImage uses? Could I simply get a reference to that image
data and send it the X server? That would make sense, right? Java is able to
keep a BufferedImage in memory and efficiently copy it over to an X window for
AWT and Swing.
The setup code became this, including a BufferedImage canvas, a Memory buffer
to hold image data, and an XImage linked to that buffer.
(defn setup [window-id]
(def x (X11/INSTANCE))
(def dpy (.XOpenDisplay x (System/getenv "DISPLAY")))
(def xgcv (new X11$XGCValues))
(def win (new X11$Window window-id ))
(def xgc (.XCreateGC x dpy win (new NativeLong 0) xgcv))
(def xwa (new X11$XWindowAttributes))
(.XGetWindowAttributes x dpy win xwa)
(def width (.width xwa))
(def height (.height xwa))
(def depth (.depth xwa))
(def canvas (BufferedImage. width height BufferedImage/TYPE_INT_ARGB))
(def buffer (new Memory (* 4 width height)))
(def image (.XCreateImage x dpy (.visual xwa) depth 2 0
buffer width height 32 (* 4 width)))
)New up was my own xput function to get the data out of the BufferedImage
and write it into the XImage buffer. The last step is a call to XPutImage
to push the XImage buffer to the X server.
Getting at the raster data in the BufferedImage took a little bit to figure
out, but I think the end result is pretty straightforward:
(defn xput [^BufferedImage canvas]
(let [raster (.getData canvas)]
(.write buffer 0 (.getData (.getDataBuffer raster)) 0 (.getSize (.getDataBuffer raster)))
(.XPutImage x dpy win xgc image 0 0 0 0 width height)))A quick test showed me that it worked! YES!
Time to whip up a simple animated demo!
You can see the titlebar says this is a Konsole window, but the content is coming from the Clojure code. It’s reasonably fast, too.
Attempting to target other windows led to the discovery that the image’s color depth had to match the target window’s color depth. Using Konsole as a target was simple good luck. But there was more to figure out to make this robust enough to work outside of this test case.
This is where the X11 Visual type comes in. X11 handles displays with color depths
from old black and white to TrueColor. So naturally the pixmaps in memory are
structured to reflect the display color depth.
A little research showed that the BufferedImage could handle a
variety
of internal representations as well.
I suspected that these just needed to be matched up. Doing that was harder to figure out than expected. The XLib color management docs were helpful, but not quite enough.
Time for brute force and some educated guesses. 24 bit color looks like it would be
Java’s TYPE_INT_RGB, right? 8 bits each for red, green, and blue for 24 bits total.
Add 8 bits for an alpha layer and 32 bits might be TYPE_INT_ARGB?
(def depth-map {24 BufferedImage/TYPE_INT_RGB
32 BufferedImage/TYPE_INT_ARGB})
;then, within (setup):
(def canvas (BufferedImage. width height (get depth-map depth)))
(def buffer (new Memory (* 4 width height)))
(def image (.XCreateImage x dpy (.visual xwa) depth 2 0
buffer width height 32 (* 4 width)))That worked. The .depth returned by XWindowAttributes is used with depth-map
to select the right image type for the BufferedImage.
It even worked when I tried the resulting uberjar on my laptop. I was shocked.
》The Demo Project
A simple proof-of-concept demo project has been uploaded to clj-xscreensaver-basic-demo on GitHub.
It contains directions on how to add the demo to the .xscreensaver config
file should you feel like trying it.
》Future Work
I have plans for a new project: XScreenBane!
A few things to figure out first:
Am I over-allocating memory for 24-bit color displays?
Is there really any need to try to be backward compatible to display depths below 24-bit?
Have I missed a faster/better way to get the data from the
BufferedImageinto the X server?What’s the best way to modularize the project?
Can I incorporate past projects without completely re-writing them?
Feedback, hack ideas, and code appreciated ;)
Contact info in the sidebar ⇒