Adventurous Accounts: The Pepperminty Wiki Android App

I've found recently that there are a growing number of things I haven't blogged about on here yet because I either haven't finished them, or I have run into difficulties. I'd like to start blogging about these projects here and there, so in this post I'm going to talk about the Pepperminty Wiki Android App.

For those not in the know, Pepperminty Wiki is a wiki engine I've built myself from scratch that is all packed into a single file using a custom build system. It's pretty stable, and I'm committed to supporting it in the long term - as I have more than 1 rather important wikis hosted using the software myself.

Anyway, as a companion to this I have also implemented a simple Android app companion that reads information from the Pepperminty Wiki Rest API. It's read-only at the moment, but long-term I'd like to build an app that can write back changes as well.

In it's current form, it works quite well actually. It's functional (although there are a few bugs lurking around). You can even download it from the Google Play Store: https://play.google.com/store/apps/details?id=com.sbrl.peppermint

I haven't really worked on it for a while though, and I'd like to talk about why that is.

Trying to implement additional features is where things get complicated though. The app is written in Kotlin, which I found much easier to use than Java (example: spawning a new thread is 3 lines instead of 30!). For some crazy reason when I was reading the guidance on building such an app, one of the guidelines was to keep the class structure as flat as possible.

This was a bad idea.

For so many reasons as well. Without using lots of files and multiple classes, a codebase gets difficult to follow and build upon. Taking this advice has severely impacted my ability to continue to add new features to the project, so a large refactor is needed to bring this under control. Add to this the complexity and ambiguity of the Android APIs themselves (how about a OnListFragmentInteractionListener, or an android.support.design.widget.BottomNavigationView? the API to display a list of things is also particularly complicated), and you can see how it can spiral out of control.

In addition, I've discovered that I have some serious problems updating it for Android 10. When you build an Android app, you have to regularly update the target platform version (I forget the exact name) - otherwise you get nagging emails from Google Play about this (I've had several already). I'm pretty sure you can't use newer APIs either if you don't keep the platform version updated.

It's been a while since I've tried to update it so I can't remember the exact error messages, but I do remember doing some pretty extensive searching around on the Internet and having no luck.

To this end, the app works in its current form quite well, but at some point (eventually) I'm going to make another attempt at writing a replacement for it. To do this though, I'd like to find some kind of framework that eases the process just a little bit - building an Android app at the moment is unhelpfully complicated and a larger time investment than I can afford at the moment with the regular maintenance required - since regular updates have to be done to keep the app updated, which don't seem to serve much purpose at all to me other than being a bother and creating extra work - since often when you update dependencies and the build system behind an Android app everything breaks, which you then have to fix.

This is different from the maintenance for Pepperminty Wiki itself, which is still regular - but constitutes fixing bugs as they crop up and occasionally implementing new features - and it has settled into a delightfully sedate pace that allows me to follow my inspiration. I find this to be much more enjoyable.

If anyone has any suggestions of alternative approaches I could try, I'm definitely interested. Comment below!

Switching TOTP providers from Authy to andOTP

Since I first started using 2-factor authentication with TOTP (Time based One Time Passwords), I've been using Authy to store my TOTP secrets. This has worked well for a number of years, but recently I decided that I wanted to change. This was for a number of reasons:

1. I've acquired a large number of TOTP secrets for various websites and services, and I'd like a better way of sorting the list
2. Most of the web services I have TOTP secrets for don't have an icon in Authy - and there are only so many times you can repeat the 6 generic colours before it becomes totally confusing
3. I'd like the backups of my TOTP secrets to be completely self-hosted (i.e. completely on my own infrastructure)

After asking on Reddit, I received a recommendation to use andOTP (F-Droid, Google Play). After installing it, I realised that I needed to export my TOTP secrets from Authy first.

Unfortunately, it turns out that this isn't an easy process. Many guides tell you to alter the code behind the official Authy Chrome app - and since I don't have Chrome installed (I'm a Firefox user :D), that's not particularly helpful.

Thankfully, all is not lost. During my research I found the authy project on GitHub, which is a command-line app - written in Go - temporarily registers as a 'TOTP provider' with Authy and then exports all of your TOTP secrets to a standard text file of URIs.

These can then be imported into whatever TOTP-supporting authenticator app you like. Personally, I did this by generating QR codes for each URI and scanning them into my phone. The URIs generated, when converted to a QR code, are actually in the same format that they were originally when you scan them in the first place on the original website. This makes for an easy time importing them - at least from a walled garden.

Generating all those QR codes manually isn't much fun though, so I automated the process. This was pretty simple:

#!/usr/bin/env bash
exec 3<&0; # Copy stdin
while read url; do
    echo "${url}" | qr --error-correction=H;
    read -p "Press a enter to continue" <&3; # Pipe in stdin, since we override it with the read loop
done <secrets.txt;

The exec 3<&0 bit copies the standard input to file descriptor 3 for later. Then we enter a while loop, and read in the file that contains the secrets and iterate over it.

For each line, we convert it to a QR code that displays in the terminal with VT-100 ANSI escape codes with the Python program qr.

Finally, after generating each QR code we pause for a moment until we press the enter key, so that we can generate the QR codes 1 at a time. We pipe in file descriptor 3 here that we copied earlier, because inside the while loop the standard input is the file we're reading line-by-line and not the keyboard input.

With my secrets migrated, I set to work changing the labels, images, and tags for each of them. I'm impressed by the number of different icons it supports - and since it's open-source if there's one I really want that it doesn't have, I'm sure I can open a PR to add it. It also encrypts the TOTP secrets database at rest on disk, which is pretty great.

Lastly came the backups. It looks like andOTP is pretty flexible when it comes to backups - supporting plain text files as well as various forms of encrypted file. I opted for the latter, with GPG encryption instead of a password or PIN. I'm sure it'll come back to bite me later when I struggle to decrypt the database in an emergency because I find the gpg CLI terribly difficult to use - perhaps I should take multiple backups encrypted with long and difficult password too.

To encrypt the backups with GPG, you need to have a GPG provider installed on your phone. It recommended that I install OpenKeychain for managing my GPG private keys on Android, which I did. So far, it seems to be functioning as expected too - additionally providing me with a mechanism by which I can encrypt and decrypt files easily and perform other GPG-related tasks...... if only it was this easy in the Linux terminal!

Once setup, I saved my encrypted backups directly to my Nextcloud instance, since it turns out that in Android 10 (or maybe before? I'm not sure) it appears that if you have the Nextcloud app installed it appears as a file system provider when saving things. I'm certainly not complaining!

While I'm still experimenting with my new setup, I'm pretty happy with it at the moment. I'm still considering how I can make my TOTP backups even more secure while not compromising the '2nd factor' nature of the thing, so it's possible I might post again in the future about that.

Next on my security / privacy todo list is to configure my Keepass database to use my Solo for authentication, and possibly figure out how I can get my phone to pretend to be a keyboard to input passwords into machines I don't have my password database configured on :D

Found this interesting? Got a suggestion? Comment below!

Java: First Impressions

(Above: The Android, Android Studio, and Java logos. I don't own any of these - nor is this post endorsed by any of the entities represented here - they are just for illustrative purposes.)

I've been using Java pretty extensively recently, as I've been doing a module on Android development at University. It's a pretty interesting language, so I thought I'd share my first impressions here. Later on in a separate post, I'll also talk a little bit about Kotlin, Google's new language they are championing for development on the Android platform.

Firstly, Android Studio has made it really easy to get started. The code hinting / autocompletion is fairly intelligent, and provides enough support that it's not too much of a bother programming in a new environment that you've never seen before - lessening the burden of learning a new language.

It seems to me that the whole build process for Java applications has been greatly overcomplicated though. It's slow, and keeps throwing random errors - especially when I've only just opened Android Studio. This non-determinism proves especially challenging for beginners, such as myself - as sometimes there's no real way to know what's gone wrong (the error messages are not particularly helpful - I've seen several languages with much more helpful ones).

There seem to be a bunch of assumptions that the developers have made too about the user's setup and programming style - leading to confusing situations in which it doesn't work - but there's no real way to know why, as there aren't any obvious error messages.

Despite this, Java as a language has some interesting features. As a whole, I can definitely see where Microsoft got their inspiration for C♯ from, as it's very similar - just without a lot of the syntactical sugar I'm used to in C♯ that makes expressing complex data structures and algorithms much easier, such as getters and setters.

Particularly of note is the exception system. In Java, if you want to throw an exception, you have to add throws ExceptionName to the method signature. Since your main activity in Android contains overridden methods at the top level, this means that you have to use lots of try..catch blocks to trap exceptions and deal with them before they bubble up to higher levels - otherwise it's a compilation error!

While this can be helpful, I've found that it can lead to awkward bugs in which an exception is eaten higher up, and the default value that's returned by the method that eats the exception causes strange things to happen that aren't immediately obvious - and it's only when you check the log that you realise what happened.....

The other bothersome thing I've found is the deeply-nested folder structure that a Java project appears to generate for even the simplest of projects. This makes it a rather difficult and involved process to find any code outside of the IDE - which I often do because Android Studio is far too slow and bulky just to check on or reference something quickly.

Finally, the last issue that concerns me are the licensing issues that have plagued Java in recent years. If you haven't heard, Google and Oracle (the company that owns Java) have been in disagreement over licensing fees which Oracle claims Google should pay them because they used Java in the making of Android (which is an open-source project). If Oracle are going after Google over licensing fees for just using a language, then what does that say about any projects I do? It's not exactly confidence inspiring, that's for sure. I for one will be keeping as much of my code library out of the Java ecosystem as possible.

Java seems to be the kind of language with a lot of history. While some of this has led to innovations that have ultimately improved the language, I feel that as a language it's being bogged down by lots of bloat and unnecessary garbage that it could really do without. C♯ has done a brilliant job so cutting through this clutter and rubbish, creating a language that both works with you and is easy to understand (except .NET Standard and .NET Core, but that's a story for another time :P).

Issues with Android Studio

I don't know about you, but I've been having a spot of bother with Android Studio - the IDE we're using for your Mobile Development ACW in which we are building an app for Android. I thought I'd document some of the challenges I've encountered in the process of installing it and using version 3.0.1 on Linux - and issues I've seen in the University labs too.

Disclaimer: This is by no means a complete list. Take advice from this list at your own risk! Additionally, any issues with the University lab machines must be reported to ICTD, whose email address you can find on your desktop background when you login.

Android Studio can't find the SDK

This issue is fairly trivial - it means that the Android SDK is probably not installed. There are two solutions here - download it through Android Studio itself, or, if you're on Linux, install the appropriate SDK package using your package manager.

Using Ubuntu it's the android-sdk package - on Arch-based distributions you'll have to consult the Arch User Repository. Don't forget to point the IDE at the location that it installed it to in the settings! You might have to hunt around a bit, but it's nothing a sudo find / -mount -iname "*sdk*" or something similar won't fix :P

Android Studio doesn't have permission to download the SDK to disk

This issue is specific to multi-user machines upon which you are downloading the SDK that you don't have administrative privileges on. The solution? Create a new directory and specify that as the Android SDK path before asking it to download the SDK for you.

After downloading the SDK, the Gradle sync fails

This is probably because the SDK version specified in the Gradle file doesn't match the one you have installed. Updating this should resolve the issue.

If not, then check the build tools version too. You can find the version it should be by opening the root of the SDK in your favourite file manager, going into the build-tools folder, and observing the name of the only folder in that directory.

Android Studio claims that abd doesn't exist

If you're on Linux, then it's likely that you don't have the Android Debugger installed. Find and install it with your package manager (it's probably called adb or similar).

If you're on Windows, check that you've set the SDK path correctly. adb can also be found in the platform-tools folder of the SDK. Also make sure that you have execute privileges on the drive you installed the adb to.

Other than that, I suspect that your installation of Android Studio might be broken, and require a re-install.

Android Studio claims that the emulator is out of date

I've has this one several times - simply press the update button when prompted (if you've got administrative privileges). I've found that the updates have made the emulator progressively more stable, so if you're experiencing issues, it's worth installing any updates it asks you about.

Android Studio claims that the "Google Maven repository" doesn't exist

Again, simply click the "add" button or whatever it is when prompted. Unlike the emulator update though, this is project-specific and doesn't require administrative privileges.

Intel HAXM errors

Another issue that I've heard of happening in the lab. I've heard that the following help:

1. Make sure that Hyper-V is turned off, as it's mutually exclusive with Intel's HAXM.
2. Delete the Intel folder in C:\ProgramData

Other various compatibility issues with the Android Studio project

If you experience any random compatibility issues when trying to open an existing project that was for an older version of Android Studio, delete the .idea folder and then open Android Studio again. The .idea folder actually just contains auto-generated files - none of which can't be replaced based on the rest of your project. To that end, I'd avoid committing it to source code control too.

Pressing start next to a virtual device doesn't do anything

I've seen this a few times - and I think it might be an Intel HAXM issue. Try reading the solution above.

Android Studio claims that the module SDK is not defined

This only happens on startup. Wait it out, and it should disappear once the Gradle sync finishes. It'll prompt you to delete a Gradle project file because it's "not part of the project", but I haven't had the courage to allow it to delete it yet :P

Errors relating to the integrated source-code-control support

I've seen many of these, but I ignore them as the external tools I use to manage my repository work just fine - and I've no desire to allow a complicated and opinionated IDE to take control over how I commit my code. If anyone knows how to disable the integrated SVN/Git support, I'm all ears!

After updating Android Studio from 3.0.1 to 3.1, all the Android API calls in my code turn to unresolved references, despite dthe gradle build being successful!

I've just experienced this (~March 31st) with a Kotlin project. The solution, according to a nice person on stackoverflow, is to delete the following folders in your project whilst Android Studio is closed:

• .idea/
• build/
• app/build/

Once done, open Android Studio again and the problem should be resolved - once it's rebuilt all it's cache files, of course.

---

That about concludes the list of issues I've seen and experienced. If you've experienced any of the above (or even a different issue) and found a different workaround, and then let me know below! Did a solution work / not work? Let me know too.

Another reminder: I take no responsibility for any damage that might happen to your computer / project / work as a result of following this suggestions. Always have backups! Additionally, as mentioned above, if you're having an issue with the machines in the University labs, you need to let ICTD know by emailing the address on your desktop background. If you don't, then they won't know about the issue!

Android app architecture: First steps and impressions

This post, obviously, is not endorsed by Google or the Android Open-Source Project at all in any way. It's just my attempt to consolidate what I've learnt about it so far.

I've been learning about Android development at University recently - this post is my attempt to consolidate what I've learnt. I'm by no means as confused as I have been in the past at similar stages of a module (take AI and compilers for example, though later on I figured it out). If you notice a mistake, please do let me know by posting a comment below, and I'll correct it.

Note that this post isn't meant to be a complete tutorial on the subject - just to consolidate what you've already learnt (or guide your learning if you're just starting out). I'd recommend taking a course at University, or reading an tutorial on the web on the subject.

Android apps, unlike a regular C or C# program, are made up of one or more activities. They don't have any particular entry point, such as the main method of a C or C# program - instead an activity is selected as the one that should be launched when the user taps the icon on their home screen. Other entry point to the app are possible too - for example services (persistent background processes) and scheduled tasks (broadcast receivers I think). Other apps can even launch your app's activities themselves!

An activity is like a single screen - it's job is to perform a single, focused task. For example a contact list, or a contact details screen.

When an app is launched, a new 'back stack' is created - into which new activities are inserted. It's this mechanism that makes the back button go back to the contacts list from the contact details screen when you press the back button on your phone. Activities can choose to launch an activity in a ne back stack if they want - though I think this only implies to implicit intents (more on these later) that target activities in other apps.

Intents are used to instruct Android as to which child activity a parent activity would like to launch, and to carry data (serialised to a string) around between activities. There are two kinds: implicit and explicit.

Explicit intents are useful when you know the exact name of the intent that you want to launch (their names are like C♯ namespaces I believe). They are most useful when you want to launch another activity that's part of your app (or an extension or your app I suppose).

Implicit intents are used when you know what kind of app you want to launch, but not what it's called. Examples of this include selecting a contact from the address book, opening a URL in a web browser, and pre-filling an email or text message for the user to send.

Fairly simple, right? Unfortunately, this is complicated by 2 things: a large number of Android versions (or API Versions) in use currently (I think Google are working on this long-term), and fragments.

Fragments are like mini-activities. Multiple fragments can be displayed at once, and can be detached / reattached to and from the screen by way of the fragment manager. This is most useful for tablets and other devices with larger screens - An activity can dynamically fetch multiple fragments and display them at the same time. Sticking with the address book / contacts theme, on a tablet one might have the contact list down the left-hand-side of the screen, and the details of the currently selected contact down the right-hand-side of the screen.

The activity is responsible for shuffling messages around between fragments (or other activities) - fragments should be completely self-contained and shouldn't be aware of other fragments that may or may not be displayed at the same time as it is.

From what I can tell, the Android ecosystem has plenty of structure to it. It's (in theory) easy to put together an app even if you haven't written any Java (I can see how Java is said to be C♯'s predecessor) before - especially with the assistance that Android Studio provides, though it does feel somewhat heavy-handed and opinionated at times. I suppose any sufficiently advanced IDE carries considerable risk of being opinionated.

I anticipate, going forwards, that the real problems will start to occur when I start considering compatibility between different Android API versions. Thankfully, I've got experience dealing with web browser compatibility issues, so I'm hoping that Android won't be much more problematic than that - especially since everything appears to be well-documented as to which API versions they were introduced / deprecated in.