Test phases for new app on play store
from the CommonsWare Community archivesAt April 22, 2020, 2:44am, Jan asked:
Hi, all. We are ready to put our first app on play store. First we plan to do some private testing from our group of users using a private listing or test area on play store (can’t remember what that’s called). My very limited understanding is that there are registered play store users out there that will test your app. Is it worth it to let these public testers test your app? Next, should we go through the step to lease time from google and/or amazon to test on many different phones (real and/or emulated) using a device farm? It’s a very simple app so testing isn’t complicated. Thanks for any help based on your own experience.
At April 22, 2020, 11:25am, mmurphy replied:
I have no idea, sorry. None of my clients have used that sort of service to the best of my knowledge, and I have not used one personally.
Those farms are less about “complicated” than “likely to have device-specific challenges”. So, for example, an app that is pretty much just forms-and-Web-services may not gain much from a device farm. But apps that integrate more with the OS (e.g., accessing contacts via ContactsContract) start to become more dependent upon possible OS version and device manufacturer differences.
It also depends on how much local testing you do. I have a “farm” of ~60 devices, so I can test lots of scenarios from here in my Secret Mountain Lair™. Most developers don’t have this much hardware. So… have you tested your app on a tablet? On a Chrome OS device? If the answer is “no”, while personally I would just buy one of each for testing, you might decide that it is faster/cheaper to use a device farm that offers screenshots.
All that being said, I have not tried one of those farms in a few years, and AFAIK none of my clients have used a farm.