share

ENGINEERING

5min read

Polidea Labs #4: Augmenting Reality on the Web

In our latest post from Polidea Labs series we were augmenting reality on an iPhone (you can read more about it here). In the following experiment we’re looking into the currently available browsers and the options they offer as far as AR is concerned.

So let’s just imagine that we can add anything to our surrounding. New cupboard next to our sofa? Just drop it there and make sure it fits. Some useful information about the cities we’re visiting? No problem. A dragon behind your back? Just turn around. We look for more information about the world we experience every day and we crave for new means of entertainment. Having our smartphones equipped with video cameras can be a real technological breakthrough and an opportunity for VR/AR to flourish.

Tools to Start

In our research we’ve found that there are a few solutions worth checking out. Among them, there are tools such as: ARLAB or AWE. We’ve decided to focus on the JavaScripy library called AR.js created by Jerome Etienne. It is a browser AR tool which allows detecting the specific markers by your camera.

This library uses ARToolKit 5—the open source SDK “which enables computers to see and understand more in the environment around them”(more here)—by the way, new ARToolKit 6 is going to be announced soon. On top of that, in order to create User Interface, we’ve experimented with Three.js and A-Frame.

Mini AR Project

1.png

The main idea was to check the potential of AR while using a mobile browser. We wanted to create the extension of our project Pixie. The aim was to put a marker on the Pixie box which is recognized by the camera and uses it to show additional information from Github project that the device is configured for.

How does it work?

We checked both tools and came up with a short analysis:

A-Frame

Since A-Frame is a framework built on top of HTML, it serves as a really good starter. It is also clean and easy to read. In this case our final solution covered a cube with git commits list (we used our open-source project) rendered on top of a marker. It is worth mentioning that we had some problems with rendering and transforming models, such as influencing one type of transformations on another.

Here’s the working example for you to try:

  • Click here in your smartphone or on a desktop (unfortunately it doesn’t work on iOS devices)

  • Point your camera to the following markers (you can print them or do it on the screen):

Kanji

Github Octocat

RxBluetoothKit

  • Take a look at the source code for this app here.

markers.png

Three.js

Three.js gives worse coding experience, mainly because you have to code in raw JavaScript. Nevertheless, it helps to understand the WebAR logic, which was hidden behind HTML curtain in A-Frame. When a marker is successfully detected in a camera view, we can add our logic. In this case, it was unintuitive to add multiple markups, so after recognizing our RxBluetoothKit’s logo, it was showing a colorful list of the project’s commits—they can be found here.

Yet again, you can try it yourself:

  • Click here in your smartphone or on a desktop (unfortunately it doesn’t work on iOS devices)

  • Point your camera to the marker (you can print it or do it on the screen).

  • Take a look at the source code for this app here

3.png

Challenges

Our AR project was fun but we had some challenges while working on it.

1. Generating markers

It wasn’t that easy to generate our own markers. There is one tool recommended by AR.js author, which requires Flash and works better on Firefox. The good thing is that you can create your own marker using some basic tools like Gimp. It is recommended to make the border quite thick and add space between the border and an image. Also, remember not to use the images that have too many details. Only those converted to 16x16 pixels worked for us but it is possible that some more generalized markers are easier to recognize.

2. Compatibility

AR.js is compatible with all the browsers in which you can access webRTC. It failed to work on Safari and all iOS ones because of the lack of webRTC. After some struggle (and an unsuccessful experiment with Flash) we eventually gave up just like the others. The good news is that the new iOS 11 and Safari 11 should support webRTC.

3. Battery

It came as no surprise that after few hours of developing and testing the phone battery was really hot and dying.

4. Rendering text in 3D

Interestingly, while rendering text mesh in Three.js or A-Frame, we didn’t see any sign of a new line. The only way to make it appear was to manipulate the container’s width or render each line separately. The second solution implies applying the same set of transformations for every line and that can, in turn, lead to mistakes.

5. Finding your place in the augmented reality

When you are adding a new object to the surroundings, you can never be sure where and how big it will be. It may also require moving your phone many times to locate this object.

6.Busy-waiting

Just remember that busy-waiting requires more attention as it may cause some complications.

Our experiment shows that web AR is a promising technology, but it’s hard to say if it can impact mobile AR development. It still uses markers while other frameworks are based on markerless solutions. It is fairly new and definitely requires a lot of improvements but still—it’s pretty amazing! It allows to build projects for every platform and that sounds really tempting!

Also, you can watch a short video with our experiment below:

share


KatarzynaSoftware Engineer

LEARN MORE

Contact us if you have any questions regarding the article or just want to chat about technology, our services, job offers and more!

POLIDEA NEWSLETTER

Sign in and expect sharp insights, recommendations, ebooks and fascinating project stories delivered to your inbox

The controller of the personal data that you are about to provide in the above form will be Polidea sp. z o.o. with its registered office in Warsaw at ul. Przeskok 2, 00-032 Warsaw, KRS number: 0000330954, tel.: 0048 795 536 436, email: hello@polidea.com (“Polidea”). We will process your personal data based on our legitimate interest and/or your consent. Providing your personal data is not obligatory, but necessary for Polidea to respond to you in relation to your question and/or request. If you gave us consent to call you on the telephone, you may revoke the consent at any time by contacting Polidea via telephone or email. You can find detailed information about the processing of your personal data in relation to the above contact form, including your rights relating to the processing, HERE.

Data controller:

The controller of your personal data is Polidea sp. z o.o. with its registered office in Warsaw at ul. Przeskok 2, 00-032 Warsaw, KRS number: 0000330954, tel.: [0048795536436], email: [hello@polidea.com] (“Polidea”)

Purpose and legal bases for processing:

 

Used abbreviations:

GDPR – Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016
on the protection of natural persons with regard to the processing of personal data and on the free movement
of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

ARES – Polish Act on Rendering Electronic Services dated 18 July 2002

TL – Polish Telecommunications Law dated 16 July 2004

1)        sending to the given email address a newsletter including information on Polidea’s new projects, products, services, organised events and/or general insights from the mobile app business world |art. 6.1 a) GDPR, art. 10.2 ARES and art. 172.1 TL (upon your consent)

Personal data:name, email address

2)       statistical, analytical and reporting purposes |art. 6. 1 f) GDPR (based on legitimate interests pursued by Polidea, consisting in analysing the way our services are used and adjusting them to our clients’ needs, as well as developing new services)

Personal data:name, email address

Withdrawal of consent:

You may withdraw your consent to process your personal data at any time.

Withdrawal of the consent is possible solely in the scope of processing performed based on the consent. Polidea is authorised to process your personal data after you withdraw your consent if it has another legal basis for the processing, for the purposes covered by that legal basis.

Categories of recipients:

Your personal data may be shared with:

1)       authorised employees and/or contractors of Polidea

2)       persons or entities providing particular services to Polidea (accounting, legal, IT, marketing and advertising services) – in the scope required for those persons or entities to provide those services to Polidea

 

Retention period:

1)       For the purpose of sending newsletter to the given email address – for as long as the relevant consent is not withdrawn

2)       For statistical, analytical and reporting purposes – for as long as the relevant consent is not withdrawn

Your rights:

 

Used abbreviation:

GDPR – Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016
on the protection of natural persons with regard to the processing of personal data and on the free movement
of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

According to GDPR, you have the following rights relating to the processing of your personal data, exercised by contacting Polidea via [e-mail, phone].

1)       to access to your personal data (art. 15 GDPR) by requesting sharing and/or sending a copy of all your personal data processed by Polidea

2)       to request rectification of inaccurate personal data
(art. 16 GDPR) by indicating the data requiring rectification

3)       to request erasure of your persona data (art. 17 GDPR); Polidea has the rights to refuse erasing the personal data in specific circumstances provided by law

4)       to request restriction of processing of your personal data (art. 18 GDPR) by indicating the data which should be restricted

5)       to move your personal data (art. 20 GDPR) by requesting preparation and transfer by Polidea of the personal data that you provided to Polidea to you or another controller in a structured, commonly used machine-readable format

6)       to object to processing your personal data conducted based on art. 6.1 e) or f) GDPR, on grounds relating to your particular situation (art. 21 GDPR)

7)       to lodge a complaint with a supervisory authority,
in particular in the EU member state of your habitual residence, place of work or place of the alleged infringement if you consider that the processing
of personal data relating to you infringes the GDPR
(art. 77.1 GDPR)

No obligation to provide data:

Providing your personal data is not obligatory, but necessary for Polidea to provide you the newsletter service

Refusal to provide the above data will result in inability to receive the newsletter service.

Profiling

In the process of providing the newsletter service, we make decisions in an automated way, including profiling, based on the data you provide.

 

“Profiling” means automated processing of personal data consisting of the use of your personal data to evaluate certain personal aspects relating to you, in particular to analyze or predict aspects concerning your personal preferences and interests.

 

The automated decisions are taken based on the analysis of clicked and viewed content. They affect the targeting of specific newsletter content to selected users registered to receive the newsletter service, based on the anticipated interests of the recipient.