see also: EQL5/QML update | draft

2020-08-12

Creating mock-up android apps without installing developer tools

Please make sure you have the latest EQL5-Android sources installed, plus the latest version of CL myApp.apk, see 'downloads' link below.

Here I'll describe how you can create fully functional android apps (and even pack customized APKs) without installing the usual android developer tools.

How can this possibly work?

For a general overview of developing interactively, please see: README myApp (just skip 'BUILD APK')

So, after developing your app either directly on the android device (using Slime), or partly on the desktop (UI files, Lisp functions), just run

 $ ./sync.sh # requires adb (see README link above) 

from the desktop (after connecting your android device via USB), which will copy all Lisp and QML/image/binary files from your desktop computer to your android device. Next time you start your app on the device, it will simply load your app from there.

To just reload all QML files after syncing (without restart), run (either in Slime or directly in the app REPL):

 > (ui) ; or (eql:ui) 

Remember that you'll need to set an objectName to every QML item you want to access from Lisp; and since auto generating the UI vars is not available if you don't have eql5 installed, you need to add them manually in lisp/ui-vars.lisp on the desktop.

Only files from qml/ and lisp/ will be synced, so make sure to put your own files under those directories.

Please see file lisp/run.lisp, this is the file that will be loaded after app startup (by default just reloading local QML files, if found). This is the 'entry point' for your own app.

For better performance, you may want to byte-compile your Lisp files on the desktop before syncing. To be able to do this using ECL, you'll need to load EQL5-symbols.lisp into ECL (in order to compile files containing EQL5 functions, e.g. for interacting with QML):

 $ ecl --load ~/EQL5-Android/utils/EQL5-symbols.lisp

 > (load "lisp/qml-lisp") ; most probably needed
 > (ext:install-bytecodes-compiler)
 > (compile-file "lisp/run.lisp") 

As mentioned, developing your Lisp code can be done using Slime (connecting to Swank running on the android device). You can also load Lisp files from the desktop by using load* instead of load:

 > (load* "lisp/my") ; see 'adb reverse tcp:8080 tcp:8080' and './web-serber.sh' 

Just remember to sync (see sync.sh) your files later, to make them permanently available on the android device.

For developing the UI you (most probably) want Qt Creator, simply to be able to sketch your UI on the desktop first. But you don't need the full, official Qt5 download (> 1 GB) from download.qt.io/archive/qt/, just install current Qt Creator that comes with your Linux distro, to be able to edit and view your QML files.

After substantial changes, you may prefer syncing (see above shell script), and restart the app on the android device (in order to reload all synched files).

If your app should not start up correctly (due to a bug), you may want to make a fallback lisp/ and qml/ directory (adapting the sync.sh script), and after syncing the fallback version and restarting the app on android, you may try to load the changes piecemeal.

For simple debugging, please see also eql:qlog (Lisp) and console.log (QML) to use together with the ./log script.

Usually the most convenient way of sketching a new QML UI is editing it in Qt Creator, doing

  File > New File or Project > Applications > Qt Quick...

preferably editing the easy to comprehend QML directly, without an UI designer, which doesn't work very well anyway.
Just press the green 'Run' button (bottom left) to show the UI. You may even use some dummy JS functions, prior to implement them in Lisp.


Quick Test

In order to test if it works for you, install either CL myApp 64.apk (64 bit) or CL myApp.apk (32 bit), ensure you have the adb tools installed on the desktop, and connect your android device via USB.

Switch to example 'my' on the desktop (EQL5-Android sources).

Now edit qml/my.qml, and just change the background color (search for color:) to, say "orange". Then edit lisp/run.lisp and add a line (at the end) like (qmsg "seems to work").

Launch 'CL myApp' on the android device, run ./sync.sh from the desktop, followed by ./restart.sh. The background should now be orange, and a message box should be displayed.


Create your own APK

(this is tested; note that you'll need to sign the new APK, see below)

If you want to repackage the APK, also providing your own icons, or change the app name etc., you may hack the APK using tools like Apktool, and you'll need to copy your lisp and qml directories to the extracted APK, in order to have them located in the right place.

So, after running e.g.:

 $ apktool d CL\ myApp\ 64.apk 

you would just copy your directories lisp/ and qml/ under CL REPL 64/assets/lib/, and repackage the APK:

 $ apktool b CL\ myApp\ 64
 $ cd CL\ myApp\ 64/dist/ 

The app name can be changed in AndroidManifest.xml (search for both occurrences of CL myApp), and the app icon (3 different sizes) can be found under res/drawable-*dpi/.

After creating the new APK (~25 MB for 64 bit, so it's smaller than the orignal one), it must be signed, otherwise it can't be installed. For this you need at least the SDK command line tools. After installing, you need to create a key (just once), like so:

 $ keytool -genkey -v -keystore release-key.jks -keyalg RSA -keysize 2048 -validity 10000 -alias comm 

Next step is zip-aligning (after creating e.g. symlinks in /usr/local/bin/ to the respective sdk binaries):

 $ rm -f tmp-aligned.apk
 $ zipalign -v -p 4 "CL myApp 64.apk" tmp-aligned.apk 

Now you can sign the app (with the above created release key):

 $ apksigner sign --ks release-key.jks --out "CL myApp 64 signed.apk" tmp-aligned.apk 

To install the new APK on the device, you can use adb:

 $ adb install -r CL\ myApp\ 64\ signed.apk 

Notes

Please note that you can't use an APK signed with a release key (like described above) for development (this is a feature, not a bug), because run-as from adb can't be used in release versions (this is to protect installed APKs from being hacked, or app data from being stolen). So, remember to always use the downloaded 'CL myApp.apk' for development, because it has been signed (on purpose) only with a debug key.

In theory, it sounds really good to be able to develop an app interactively on the mobile device. But there is a 'but': unfortunately, Swank hangs/freezes sometimes, just randomly. In my experience this may happen after a few seconds, or after 10 minutes, or much later. But it really is a show stopper when it happens. I experienced this mostly during TAB completion, or while reloading QML.
Of course your personal experience might (hopefully) be better than mine, and maybe it also depends on the Slime/ECL version, so I don't want to discourage you from making a better, personal experience!

I generally develop as much as possible on the desktop, only fine tuning some UI part directly on the mobile device. If Swank hangs, you can't just reconnect, sometimes you'll need to restart the app on the device (or even restart Emacs on the desktop). I have no clue why this happens, maybe some thread race condition (just a poor guess).

The APK size of 'CL myApp' is bigger than usual because it's a debug build, and includes many QML modules (for playing around) and also Slime. Installing all dev tools and creating your own APKs (as described in EQL5-Android READMEs, including only QML modules you actually use) would most probably result in significantly smaller APK sizes (say less than 20 MB instead of almost 30 MB).


Conclusion

I'm really grateful that someone on the #ecl irc channel asked for a tutorial how to accomplish the above. I only now realize how this can speed up development, even without using Slime etc.
On my phone (combining the sync and restart scripts) pushing the changes to the phone and restarting the app takes only about 3 seconds! That's a huge advantage for interactive development, even for newbies, who don't know yet how to use Slime effectively, or how to reload QML:

Now anyone can push their changes and have everything reloaded almost instantly!