Welcome to a new BetaKit bi-weekly series designed to help startup developers. Each article, TribalScale’s tech honchos tackle the tough questions facing devs today. Have a question you would like answered? Tweet them with the #askadeveloper hashtag, or email them here.
By: Lionheart Xiong and Arjun Achatz
Augmented Reality (AR) was all the rage at this year’s SXSW Conference and Festival and the technology is likely here to stay. Technologists have been enhancing and refining AR’s capabilities, and businesses are starting to notice that AR has the potential to completely disrupt how we interact with digital products and see the world.
As AR is being looked at by technologists and industry leaders it is becoming more and more prominent.
With AR emerging as a sticky technology, global tech companies like Google and Apple are developing their own AR development platforms: ARKit for iOS and ARCore for Android. The release of both ARKit and ARCore is important because they give developers the opportunity to experiment in building AR-enhanced experiences without investing in expensive hardware.
But we’re still in the early stages of AR learning and development. As a reminder to engineers, when developing, it is most important to consider what experiences are native to AR. For example, if an app supports AR, but the content it provides is not very different if it were displayed in 2D, then AR might not be necessary. Remember, you want AR for a reason and if there is no real value-add, your app risks being gimmicky.
We wouldn’t suggest adding static features for the sake of it being AR capable; you want AR to give and bring meaning for the user (for example: a game in which the player is plunged into water would benefit from AR and added dimensions). Keep in mind that AR is about meshing and merging real and alternative environments. AR begins to add value to the app once there is added spatial information.
ARKit and ARCore overview
ARKit is Apple’s set of software development tools (released in June 2017) that enables developers to build AR apps for iOS. ARCore is Google’s analogous set of tools that was built off Google Tango (formerly Google’s AR platform). Unlike Tango, ARCore doesn’t require specialized and expensive hardware for an accurate AR experience. ARCore can run on millions of Android phones, and this level of accessibility makes it easier for developers to add AR experiences into their app. Apple is also training and using ARKit as an accessible entry-point for 3D development — plus Apple is rumoured to release Apple glasses. To date, users need iOS 11 to take advantage of the ARKit built products, and since Android works with many OEMs, ARCore is limited to qualified phones with Android 7.0 and on.
In terms of functionality, ARCore and ARKit aren’t very different. Both are good at detecting horizontal surfaces, but unlike ARKit, ARCore can’t map vertical planes (although that is something we will likely see in the future). ARKit has tracking capabilities that produce a precise function of the device’s motion and position in space. On the other hand, ARCore has more reliable tracking recovery, and can better map a person’s surroundings than ARKit. Matt Miesnieks, CEO at 6D_ai has a helpful post on the benefits of each.
ARCore and ARKit are welcome advances for AR and app developers. But as mentioned, we’re still in the early stages of using ARKit and ARCore to figure out what we can and can’t do with AR. Many developers are practicing placing content on 3D surfaces while thinking about how AR can provide additional value to the content being presented.
Handy ARKit and ARCore development tips
There are a few frameworks for AR development (which are also used for 3D games). Unity is a game development platform that we use to develop and deploy to different platforms (both iOS and Android), so a developer can use both ARKit and ARCore with Unity. SceneKit is a native Apple framework and is a tool that iOS developers use on XCode (Apple’s developer platform) to deploy on to iOS. As always, there are pros and cons for each, and it really comes down to what you want to achieve with your product.
When building your app, we encourage developers to think about placing content in 3D rather than in 2D — since AR changes the way users consume and experience digital content, it only makes sense that you change how you build. Consider that models render to light, so if you set your model with a specific lighting and then dim the lights, the app will also dim. So, when developing, make sure you factor in lighting and encourage a specific light setting in app instructions.
Also, it may take time for the AR app to fully and correctly detect surfaces, so remember that setup may take a while for a user. The user onboarding experience is therefore crucial, otherwise, it could break or kill the app! We suggest you carefully guide the user through setup and calibration with clear instructions for the user (a bot could be a great approach). Check out the IKEA AR app — which uses ARKit — it’s great for user onboarding.
There are still some limitations to developing with ARKit and ARCore. For one, they are not geolocation-aware or real-world object aware platforms. For example, Pokemon GO, while highly popular, doesn’t have building awareness and cannot map real-world architectures. Also, depth perception in ARKit and ARCore lags, meaning that detecting objects that are more complicated than a flat surface (a chair, for example) can be challenging. If you’re looking for more advanced recognition and awareness of space, we suggest using more powerful hardware — some advanced glasses are: HoloLens and Vuforia. To get a more interactive experience from ARKit, you can also combine the existing VR hardware with the ZED Mini.
The future of AR development
As ARKit and ARCore are relatively new advances, we expect to see more developers working in this space, striving towards more real-world aware and geo-mapping capabilities. Currently, some startups are working to improve complex object detection in AR (see Placenote), and some are beginning to map real world architectures (Blippar AR City and 6D.ai).
As AR is being looked at by technologists and industry leaders it is becoming more and more prominent. In response, ARKit and ARCore are making impressive advances in the AR space possible — and feasible — for developers. We will likely begin to see a greater range of offerings and more value in AR apps. For the latest updates in the AR world follow the AR and VR Facebook Group, the ARKit page on GitHub, and Google AR on GitHub.
Arjun is an Agile Engineer at TribalScale. Arjun focuses on Android Development for all devices and has a particular interest in AR. He has a Bachelor’s Degree in Mathematics from the University of Waterloo.
Lionheart is an iOS Engineer at TribalScale and he focuses on ARKit and Unity. He has a Bachelor’s Degree in Computer Science from the University of Toronto.