asgaard

Wellburrito

Fri, 01 Jan 21 19:57:44 +0000

I'm very pleased that we have released the first public version of Wellburrito.

Wellburrito is more than just a mood tracking app, it's a full wellbeing app focusing on mood and custom metric tracking as well as cognitive behavioural therapy (CBT) exercises.

Technically, Wellburrito is developed using the Ionic framework. Data is intentionally stored purely locally on the device, i.e. without a server, for the user's privacy. For this we use an SQLite database, which is optionally backed up to Google Drive on a regular schedule. Android does actually take care of restoring the database on app reinstallation through Google Play, as well, if the user has it enabled, via Android's auto-backup feature.

On the mid to long term roadmap is reducing dependency on things beyond our control that aren't really of the quality or flexibility we'd like. For example, we'd like to expand the reminders system such that users can interact directly with reminders. When a medication reminder comes up, we'd like to be able to press a little "Yes, I've taken it" button. When the mood journal reminder comes up, it'd be good if the user could select a mood within the reminder. Those things are currently not possible because the reminders library we're using doesn't support it. Unfortunately, getting the control we need means writing our own library.

Similarly, we'd like charts to better fit with the look and feel and branding of the app, but we've pushed the current charting solution to its boundaries.

There are a few little things like this which seem like relatively small features but require a large amount of work. We've pushed these back to a future version for the moment, in favour of getting something functional in front of users.

In terms of features, we'd also like to implement much deeper analysis. Currently we have superficial analysis and reporting of trends and correlations, but there is a lot more we could do with this. Again, this is a 'version 2' feature.

You can read more about the Wellburrito wellbeing, mood tracking and mindfulness app on the official page.

You can download the Wellburrito wellbeing app on Google Play.

Ionic Native

Fri, 01 Jan 21 10:21:58 +0000

Ionic is a framework for developing mobile apps. I've completed a few successful projects using Ionic. I like Ionic. I think that as far as cross platform mobile development goes, Ionic is pretty good.

Ionic was built on Cordova, and has been gradually moving away from it with Capacitor. I like Capacitor better than Cordova. I think they've done a really good job.

Ionic Native is a set of Cordova plugins that they've provided a wrapper for. If you look at the list, it looks very impressive. If you try to use any of them, though, it's a toss up as to whether you end up with something production standard. Some of the plugins there shouldn't be on the list because they're just broken.

The Local Notification plugin is a good example of this. Currently, it's pointing at the underlying Cordova plugin version 0.9.0-beta2. At the time of writing, this is around three years old. The most recent version is 15 months old. So not only is the plugin not actively maintained, we're also out of date even by the plugin's standards.

Unfortunately it doesn't work very well on modern Android operating systems. Trying to schedule a repeating notification causes the plugin to go into an infinite loop until the OS steps in and kills it, so the workaround is to set a notification with a large 'count'. This causes very slow interaction with the notification in future, presumably because the operating system is dealing with a notification instance per count. This is something you can work around by scheduling, say, 100 days in advance and making sure your app reschedules when you're getting close to the end of the 100 days. You shouldn't have to do this, but it's an option.

However, there's a much more serious problem in that the plugin can't schedule notifications that persist past the device rebooting. There are a few use cases where this may be acceptable, but for a general notification system, this is a deal-breaking limitation that you don't find out about until testing (and physically testing notifications is tedious and error prone enough already).

Then there's the in app purchases plugin. The IAP plugin is bizarre. When dealing with subscriptions, there is no way of simply querying whether the user owns a subscription, because the 'owned' field isn't reliable. When the plugin starts up, a subscription goes through a variety of states, but always starts with owned = false. Maybe it reaches owned = true, maybe it doesn't. The point is that owned = false is never a useful piece of information because there's no guarantee it's correct. This means that there is no completely reliable way of determining whether a user's subscription has expired. You have to look at the subscription billing period and be aware of when it may have expired, and if you go a certain length of time with successful communication with the plugin but without seeing owned = true you then assume that it must have expired. This is absurd, and I would be extremely surprised if the underlying billing library exposes subscription information in this way.

Ionic should not be providing wrappers (and exposure) for these plugins unless they meet a certain standard. It's a waste of developers' time to attempt to use sub-standard plugins and will fuel a perception that it's difficult to achieve production quality software using Ionic.

Update: Another one - Ionic Native Audio doesn't work with Capacitor because it hard codes audio assets as existing within the www/ directory. In Capacitor, this directory has been named public/ so the plugin can't find any of your audio assets. Again, why is this being advertised by Ionic, with Capacitor instructions?

npm is terrible

Wed, 02 Aug 17 08:49:46 +0000

It's hard for me to believe in the quality of a piece of software that behaves like this:

$ npm install [package]
  npm ERR! Please try running this command again as root/Administrator. 
$ npm install [package]
  (OK)

I'm sure about five years ago npm was a useful and reliable piece of software. Perhaps I remember incorrectly.

I had another tale of npm woe last week when after runing `npm install`, my node_modules/package-name was a symlink to none other than node_modules/package-name. For some reason, downgrading npm to version 3 (from version 5) made it all work properly.

I groaned today when I had to install yarn, yet another package manager, to install something else, but the sooner something sane replaces npm the better.

[...]

Enabling internet permissions in a Cordova/Ionic app

Sun, 30 Oct 16 20:34:42 +0000

If you try running a Cordova/Ionic app with few plugins, you'll probably be very confused when you try to create an HTTP request. It'll work fine in the browser, but on a device it'll fail.

If you hook it up to the Chrome developer tools and inspect the webview, it'll show Status: (failed) and Type: Pending.

The answer to this is that you need to ask for access to network permissions.

It seems like you should be able to define your app's permissions in the config.xml file, but this doesn't seem to be the case. However, plugins can declare permissions in their plugin.xml file.

So, I created a plugin which does nothing else than defines a plugin.xml which asks for network permissions.

To install it, use:

cordova plugin add https://github.com/markwatkinson/cordova-plugin-internet-permissions.git

[...]

Your APK's version code needs to be higher than 100008.

Sat, 15 Oct 16 18:59:40 +0000

When trying to upload my Android app Crossword Solver CS to the Play store, I have been encountering the error:

Your APK's version code needs to be higher than 100008.

The app is an Ionic/Cordova app and Cordova handles creating that version code.

The 8 on the end makes little sense because the version I had there already was v1.0.0. I checked through git and my config.xml always had the version as v1.0.0.

Cordova claims that the version code will be generated automatically from the string and will be equal to: versionCode = MAJOR * 10000 + MINOR * 100 + PATCH.

For some reason it has followed this formula, then concatenated an '8' on the end making the end result an order of magnitude too big.

As far as I can tell this has come about because I used a Linux VM to generate the initial release and a different machine to generate the update. The latter has a newer version of Ionic/Cordova, which presumably does not have the '8' bug.

How do you fix this? Well, you need to bump up one of the components to multiply your end result by 10 and offset it by 8.

So I have gone from v1.0.0 to v10.0.9, which seems a bit ridiculous.

Ionic JavaScript thoughts

Sun, 21 Feb 16 21:48:58 +0000

First, Ionic is a JavaScript framework for building mobile apps. It's built on Cordova (the free version of Phonegap) and AngularJS and gives you a set of tools for compiling your code into a native app with a webview. It comes with some AngularJS services and directives for emulating mobile functionality as well as handling effects like screen sliding transitions.

In general I was surprised by just how native the running product feels. You have to do some work to keep it that way, of course, but for a simple forms based app it's very achievable to end up with something that looks and feels native. Savvy users will spot the difference, but most probably won't. The CSS graphics performance on a modern iOS device (say iPhone 5+) is very good and the animation effects are smooth. On Android (5) things are a bit choppier, but still acceptable.

The other major positive is that it's much more convenient to develop things locally in a browser, and use all the debugging tools that come with it, than to go through the much longer debug cycle of deploying native code and trying to use an unfriendly debugger that frequently just dumps you to a page of assembly code. I definitely find HTML, CSS and JS to be a faster and more pleasant development environment than Objective C and Xcode's UI designer (I can't speak for native Android development).

There are also downsides to Ionic:

Native functionality is provided through plugins. These are community made and the quality is wildly variable. It is easy to end up digging through the source of these yourself. On iOS, these will occasionally conflict with each other due to bad namespacing by the author. Most plugins will support iOS and Android, but support for other mobile OSes gets patchy quickly.

Although it's write-once-run-anywhere, you do end up spending a lot of time fiddling with OS specific CSS bugs. iOS's HTML renderer is far from perfect, but Android's is pretty good!

The build process is a bit annoying and we've found it hard to lock everything into our version control system due to some things about the way ionic is structured, especially in regard to mixing Linux (which can't build for iOS) and Mac machines in our project. We've also had numerous things break unexpectedly which later turn out to be due to some detail of the build process.

Ionic's JavaScript libraries are of variable quality. I recommend simply not using $ionicPopup at all and instead writing your own popup library (or finding another one), because the one built into Ionic has some catastrophic bugs like the fact that displaying two popups at once will render the first unresponsive, which means the user has to restart the app. There's also a timing issue with rendering a popup during a screen transition which will cause the popup to not have any buttons (so again, the user has to restart the app because they can't close it). The developers do not appear responsive in fixing these issues. Other services have smaller bugs which are less problematic, but still don't inspire a huge amount of confidence.

Overall: pretty good, but not without problems.

Bootstrapping (or not) a visual designer

Sun, 29 Mar 15 14:28:42 +0000

I worked the previous few years on a project to create a new form of IDE. The IDE itself was not aimed towards programmers, it was more of a drag and drop form of development. I won't discuss the details or merits of that approach, instead I am going to focus on an interesting consideration we came across in trying to bootstrap such a system to use aspects of itself for its own development, especially when just trying to get to an MVP (minimum viable product) release.

Imagine that you have a form designer and a properties panel and you're not far off. As an example, imagine Visual Studio's Winforms designer was created using Winforms (which some research suggests it is not).

In our system, the notion of a form was very important for the user. We chose to implement the form's properties panel as a form, which is a chicken and egg bootstrapping problem. It's similar in principle to bootstrapping a compiler, but the realities are very different because a compiler is written by programmers for programmers, but in our case our IDE was written by programmers for non-programmers.

Some big problems came out of this decision, which weren't obvious until they happened.

Firstly, the components you expose to the user are initially not interchangeable with the components you need internally, even though they look the same. In a sufficiently advanced system they would be the same, but that only comes about when your system is advanced enough to be able to create large chunks of itself (much more holistically than just an individual entity like a form). Up until that point you must have hard coded behaviour standing in for the bits that your system is not powerful enough to create. This hard coded behaviour was surprisingly divergent, so the code becomes difficult to manage because you are conflating the components you as a developer need and the components your users need. The problems you encounter are all in the code, so development and maintenance and compatibility becomes problematic and time consuming, but the root cause is the spec wants two things to be equal when they are simply not. This is a bootstrapping problem, so it could be solved by keeping the bootstrapping process going on longer, but I don't really want to go there because you could take a short-cut with a native properties panel and still be able to bootstrap correctly later on.

Functionality and scope creep is a problem. The form you expose to the user can be quite simple in an MVP, but your own GUI needs to be complex enough to allow the user to interact with its designer. So if you want a context menu, you need to implement a context menu on your users' form. A context menu on a native panel is easy to implement and doesn't really pose any problems. A context menu on your own representation of a form needs you to implement the notion of a context menu and expose that somehow, which increases the complexity of your requirements and greatly slows you down. In short you are facing problems that need not be there.

A surprising outcome is the increased difficult in debugging the IDE. Looking through a stack trace while a user's form is running results in multiple forms in the stack, one of which is the user's and several of which are your own. This becomes extremely confusing when spending long periods of time paused at a breakpoint trying to visualise your program's state by jumping up and down the stack. This seems like it should be a secondary consideration, but in reality projects that are hard to debug are projects that are slow to develop. Clear separation is good for debugging.

Finally, this approach exposes you to performance problems much sooner than you would otherwise need to address them. Suppose a form take 3 seconds to boot up when the user presses 'run' due to some overhead in some complex routines. That's not great, but it's acceptable because the user doesn't press 'run' very often and they don't really expect or need it to be instantly responsive; a few seconds is fine. Now suppose every selection mouse click in your designer leads to two forms being rendered in your properties panel - suddenly selection takes 6 seconds, and that's a dreadful experience for the user, especially when compared to a completely hard coded panel that would fully render in milliseconds. You can solve this, but you shouldn't have to.

The running trend here is that everything becomes hard long before it needed to become hard. Although this kind of thing is intellectually interesting, it falls into the "being too clever" category and is directly opposed to getting an MVP up and running.

How (not) to write regular expressions

Sun, 15 Feb 15 16:17:07 +0000

A few days ago there was a regular expression building library featured on Hacker News: https://news.ycombinator.com/item?id=9033146

Its premise is that regular expression syntax is more confusing than using an object oriented method chaining approach.

The comments are overwhelmingly positive, and the library itself has a lot of attention on GitHub, and I find this strange because using the library appears more complex than just learning regular expression syntax to a fluent level and writing them directly. While something like Linq for regular expressions would be very interesting, this is not it. This seems to fall into the trap that it makes trivial things easy to do and hard things harder, which isn't very useful.

Let's try rewriting a fairly simple PCRE regex to match a doubly quoted string with a backslash escaping scheme into an object oriented construction syntax.

/ " 
  ( [^"\\]+ | \\. )* 
  ( " | $ ) 
/ xs

It's not particularly easy to read, but it's also not particularly hard. In theory it should be a good candidate for simplification by an alternative construction method. To convert this into a chained construction, after a quick glance at the API docs, I'd expect to write something like this:

r.find('"').then(
  r.maybe(
    r.anythingBut(r.find('\\').or('"')).or('\\.')
  ).then(r.find('"').or(r.endOfInput())
)

This is hardly an improvement. It doesn't end up reading anything like normal English because English doesn't really handle the nesting, so we have swapped the small amount of PCRE syntax clutter for a larger amount of English and OO syntax clutter, which in comparison, makes it extraordinarily hard to scan and get a feel for what the expression actually matches. This would only get worse on genuinely complex expressions, which this one is not. It's not clear whether the capturing groups are preserved, so we might have to add more clutter for those, and then we'd only end up deeper in a mess if we had to start considering details like lazy vs non-lazy token consumption and explicitly preventing backtracking.

In summary: regular expressions are useful; if you need them, learn them.

When should I use a JavaScript compiler, and which one should I use?

Sat, 20 Dec 14 13:40:47 +0000

There are a few different options for Compile-to-JS languages and here I'm going to outline the main contenders as of late 2014.

Plain JavaScript

Let's start with the basics. A lot of people who do not understand plain JS berate it for many things, but in reality its main practical drawback is that it becomes unwieldy on larger projects (like any dynamic language), and in such cases, having a checking compiler will cut down on a lot of unit testing. Plain JavaScript is fine for small projects, it becomes problematic as your code grows; the point at which that happens depends on the skill and discipline of your developers.

Regardless of what you choose, you're going to be working indirectly with JS and you should learn it to a fluent level.

CoffeeScript

CoffeeScript (CS) was very popular when it was released but popularity has since waned. Its only feature is syntax, which implies that there is something seriously wrong with JavaScript's syntax (subjective), but fixes it by making it completely different (Ruby-ish) instead of identifying exact problems and improving them. It is now quite surprising to come across CoffeeScript code, which suggests that people have generally rejected CS as an improvement (an opinion I agree with). It does not really help with the fundamental problem of code management, because it's just another dynamic language. Avoid.

Dart

(Google's) Dart is a strange one because although Dart code can compile to JavaScript, it's really designed to be run on a virtual machine, which only Chrome implements. For this reason, it is not worth evaluating further. Avoid unless you have a particular reason not to.

Google Closure Compiler

Closure is a checking compiler from Google, and it is interesting in that it works on pure JS code, i.e. it's not a separate programming language. Code written for Closure needs comment type annotations (which are quite ugly and cumbersome, but easily written if you are disciplined) to make use of the compiler's features. The compiler does useful things like checking that the methods you are calling really exist, instead of leaving you to find out at runtime.

The main downside of Closure is performance; there is no incremental compilation which means if you change one line of one file you have to recompile the whole thing, and it can take several minutes. I should stress that for development this makes little difference - you tend to develop using your raw code, not the compiled code, so invoking the compiler is something you do before committing code, not (always) before trying it out locally. To make full use of Closure all external libraries need to have type definitions; for popular libraries these are easy to find and it is not too hard to write these yourself (but it is an extra requirement).

Google's investment in AtScript (see below) may have implications for the future of Closure, but the compiler itself is mature and reliable as it is.

TypeScript

TypeScript (TS) has been around for a few years now and only recently seems to be finding its niche, although growth is still slow. TS is a superset of JS and adds some useful syntax, but its real value is in allowing you to write strongly typed and checked code, similarly to Closure, but its standardisation allows IDEs to give you things like code completion. If your average foray into JavaScript is in writing a 50 line jQuery plugin, TypeScript is completely overkill for you. On the other hand, if you are working on a JS project with multiple developers and hundreds of files, TypeScript is very useful, although it is only recently that the tooling has become mature enough to handle this scale.

The tooling (via Visual Studio) is (now) both good and stable, and the compilation step in development is fast and largely transparent. If you are using an MS environment and have access to Visual Studio, you should absolutely strongly consider using TypeScript. MS claims that tooling exists for non Windows based platforms but I cannot speak to its quality (and frankly, I'd be sceptical).

Note that TS requires type definitions for external code, like Closure.

TS is a bit of a risk in that growth appears slow and MS might see that as a reason to discontinue it.

AtScript

AtScript is superficially not dissimilar to TypeScript, but at the moment the project is too immature to seriously evaluate how it practically differs, and because of its immaturity it would be unwise to start using AtScript any time soon.

Summary

In summary, your choice for "which compiler do I use?" depends on project size, project length, and number of developers. I'll condense this all down to lines of code to get the point across. In order of preference:

Project Size	Approx #lines of code	Choices
Tiny	1k	Plain JS
Small	10k	TS, Closure, Plain JS
Medium to very large	>10k	TS, Closure

Peripheral vision and security

Tue, 02 Dec 14 18:46:25 +0000

Here's a bug report from the Composer project: https://github.com/composer/getcomposer.org/issues/76

The report (rudely, but) correctly, points out that the recommended way of installing the script, by getting it via curl and piping it into an interpreter, is a security risk in the event that the source gets hacked and starts serving malicious code. That's true - I agree.

But hang on a minute, Composer is a package manager for a programming language. The whole point of Composer is that you download other people's code and then include it straight into some executable of your own! You could literally be pulling in 50 different packages, each of which suffers from the exact same vulnerability - if the source is compromised, you are executing very untrusted code (instead of merely we'll trust it because it's convenient code).

I'm not saying you shouldn't use package managers, nor am I saying that signing scripts/executables is a bad thing, but I am saying you should be a bit sensible about evaluating security risks.