eNotify Quality Report
eNotify provides private and secure direct local notification for incoming email and SMS messages.. Highly configurable, eNotify supports complex rules with custom alert sounds for different accounts, senders, subjects, recipient addresses, battery events, and cellular data events. Easily silence day-to-day background noise and stay aware of those important emails!
It is available for both Android and iOS. This article looks at the Android version.
eNotify Crash Report:
Crashes:
As far as the crashes, I'm very pleased with only having 2. It is really good considering the app has been under active development throughout its entire lifetime [~3 months]. It has had over 40 upgrades including 2 rounds of major new feature sets. It has processed tens of thousands of emails some with bespoke drivers. To achieve this without sacrificing quality has been a remarkable result for us.
ANRs
What's an ANR? It stands for "Application Not Responding". This can happen for example when the app does something that takes too long to complete. In this scenario, it can prevent the UI from responding to the user. You will generally see this after the device has been stuck for 5 seconds or more. When this happens, the user is shown a dialog with options "Force close" or "Wait".
eNotify has not had any ANRs thus far in either the full or lite versions. This is really unusual considering the amount of processing and network activity that this app does. The app is essentially a full email client in terms of the processing. To have no ANRs shows that our design and internal structure is solid and functioning reliably.
In general, the app avoids ANRs by doing most of its work in a service running background "threads". This prevents the main UI "thread" from getting stuck.
eNotify Lite Crash Report:
eNotifyLite has has a total of 16 crashes. It processes far more email than the paid version. The paid version can do between couple thousand and ~10K per week at the moment. On the other hand, the lite version process tens of thousands of emails each week..
Of the crashes, 4 occurred in the MOPUB advertising library and 1 in the TAPJOY library. So 31% have occurred in third party advertising libraries...Luckily MOPUB is open source, so I was able to download their source code and fix it.
A further 3 crashes occurred as a result of corrupt build after I upgraded the Android development tools. This was totally my fault. I should have been more careful before deployment. So I've upgraded our procedures to always include a "smoke test" for any new version of the app no matter how trivial the changes.
The remaining 9 crashes (56%) are real human programming errors by me. These are the ones that were perhaps less avoidable if you allow for humans making mistakes! Grudgingly, I do make them :) But overall, the quality of eNoitfy Lite is very good thus far.
Staged rollouts:
This is a helpful new feature added to the Android market. You can release your app to 5% of users instead of the entire user base. This is a God send for controlling quality. Once your're happy with the release, you can increase the percentage. The only negative aspect is that development console statistics are only updated once per day. So it can take a full day before you get enough feedback to increase the percentage.
When making significant changes now, I only release to 5% of the users until I'm sure the new version is sound. This isn't a replacement for testing of course, but it saves my bacon in the event that anything gets through.
Conclusion:
I'm really happy with the quality of eNotify. Quality is our only option. If we release builds that crash and do not work our user base would come down on us like a ton of bricks. This is an app that's informing people of important messages. Many of the users are first responders and people whose jobs entail significant responsibility. It is our responsibility to deliver a good quality app that works reliably.
Update: August 7th
Now that we are done with the major feature sets the quality is excellent. Our growth rate, continues to increase month on month, but we've only had 2 crashes and 1 ANR over the past 4 weeks. I know! Our first and only ANR. I was really proud of the zero record. It was painful to lose it!! :)
- One crash in mopub advertising library while displaying a flash ad
- One crash in the app which has been fixed
- This particular ANR is a Android platform defect that has an existing ticket raised against it
We continue to be careful with our roll-outs. It takes up to week now to get simple features to the users which is a shame. but quality control is essential. The bottleneck is the Google console. It only updates once per 24 hours so it take a lot longer than it otherwise would to do incremental roll-outs. It would be nice if it updated more frequently during staged roll-outs.
eNotfiy:
https://play.google.com/store/apps/details?id=com.hermes.enotifylite
eNotify Queries:
android_support @ verietassoftware.com
Glen Cook:
http://ewhizmobile.com/
