share

ENGINEERING

7min read

How to respond to any messaging notification on Android

If you are an Android user you are probably familiar with the awesome app called Pushbullet. If not: it’s an app that syncs all your notification across devices, including PC & Mac. Since its February update it allows you to respond to messaging apps such as Facebook Messenger, Hangouts, Telegram and a couple more - awesome? Sure is! Sadly there is no easy-to-implement API to do so. Basically, each messaging platform requires us to implement clients for all of them - not a good solution. As a dedicated user that fell in love with that feature, our engineer Michał Tajchert started to dig around in that topic after his curiosity was triggered by a “How does it work?” question on Reddit. This feature must’ve been connected with Android Wear API, even the authors said that. Small reminder – Android Wear is the platform for Android powered smartwatches. To be more specific, we are not talking about brand new Data API or Messaging API for smartwatches, but a very small addition to the already existing Notification. Since Android Wear was introduced, we can add a small object called WearableExtender that adds some features dedicated for smartwatches – like actions on pages, background, voice input etc.

RemoteInput remoteInput = new RemoteInput.Builder(EXTRA_VOICE_REPLY).build();

NotificationCompat.Action action =
        new NotificationCompat.Action.Builder(...)
                .addRemoteInput(remoteInput)
                .build();

NotificationCompat.WearableExtender wearableExtender = new NotificationCompat.WearableExtender().addAction(action);

NotificationCompat.Builder notificationBuilder = new NotificationCompat.Builder(MainActivity.this).extend(wearableExtender)

Looking at the code above we can see that we have RemoteInput (an object that collects voice response), that is added as an Action to our WearableExtender object which finally is added to Notification itself. So if you have an app that shows a notification and waits for the user’s text response (like any messaging app) this is the perfect solution for you. Most popular apps have this already implemented, there are only very few that still lack this - like Skype which is planning to add this in a matter of weeks.

How to catch notifications?

On Android it is quite easy, just implement your own service that extends NotificationListenerService and have methods onNotificationPosted() and onNotificationRemoved() which will be called each time any notifications appears or is canceled. Security to access notification is done by prompting the user with a full screen settings screen with a list of apps that can have access to your notifications and the user can check or uncheck any of them. This is bad for UX as it is full screen (so the interaction needs to go out of our app) and there is a list of apps instead of asking permission for our app specifically in a dialog, which would be a much better pattern.

But now lets get back to code. So we get called each time a notification is posted with an object StatusBarNotification which is a wrapper for Notification itself. So probably by now you suspect that there is a method for example statusBarNotification.getNotification.getRemoteInputs() or .getActions(), which in second case is even true but it will return Actions objects but not from WearableExtender but from default notification - like buttons to “like” in Messanger case, so no RemoteInput here.

Lets see how we can access WearableExtender

First way of doing so was actually posted by some user online as extracted from the source of Pushbullet, with some code fixes to make it work. This is what it looks like:

Bundle bundle = statusBarNotification.getNotification().extras;
for (String key : bundle.keySet()) {
    Object value = bundle.get(key);

    if("android.wearable.EXTENSIONS".equals(key)){
        Bundle wearBundle = ((Bundle) value);
        for (String keyInner : wearBundle.keySet()) {
            Object valueInner = wearBundle.get(keyInner);

            if(keyInner != null && valueInner != null){
                if("actions".equals(keyInner) && valueInner instanceof ArrayList){
                    ArrayList<Notification.Action> actions = new ArrayList<>();
                    actions.addAll((ArrayList) valueInner);
                    for(Notification.Action act : actions) {
                        if (act.getRemoteInputs() != null) {//API > 20 needed
                            android.app.RemoteInput[] remoteInputs = act.getRemoteInputs();
                        }
                    }
                }
            }
        }
    }
}

What just happened here? Firstly, we extracted Bundle of notification as this is where the WearableExtender parameters are located, then we searched for a key with the value of “android.wearable.EXTENSIONS” which give us access to all Android Wear specific actions - such as pages and … RemoteInput which we are looking for! To find it we needed to iterate over all actions. And this actually works, but the code does look ugly and “hackish”.

WearableExtender

Our development setup, we have found emulator working very well and much faster than a real device (especially via Bluetooth connection).

Can we do better than that?

Yes! One line to get WearableExtender, how cool would that be?

WearableExtender wearableExtender = new WearableExtender(statusBarNotification.getNotification());

Ok, so now lets just extract Actions and from them our beloved RemoteInput.

List<Action> actions = wearableExtender.getActions();
for(Action act : actions) {
if(act != null && act.getRemoteInputs() != null) {
                RemoteInput[] remoteInputs = act.getRemoteInputs();
            }
}

Ok, but what else do we need? We definitely need PendingIntent to send our results back to the app that triggered the notification and it would also be good to keep the Bundle as it might contain some extra values such as conversationId. Let’s do it!

notificationWear.pendingIntent = statusBarNotification.getNotification().contentIntent;
notificationWear.bundle = statusBarNotification.getNotification().extras;

Now we are ready to send it back! The approach in our sample app, to speed up the development process of the POC, was to pass our data via EventBus to Activity where it would be saved on Stack, and when the user clicked “Respond to last” to just pop the last item and respond to it.

Filling the RemoteInput

Probably the most interesting part is how we can fill an object that normally takes voice input with our fake text. It is not that hard.

RemoteInput[] remoteInputs = new RemoteInput[notificationWear.remoteInputs.size()];

Intent localIntent = new Intent();
localIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
Bundle localBundle = notificationWear.bundle;
int i = 0;
for(RemoteInput remoteIn : notificationWear.remoteInputs){
    getDetailsOfNotification(remoteIn);
    remoteInputs[i] = remoteIn;
    localBundle.putCharSequence(remoteInputs[i].getResultKey(), "Our answer");//This work, apart from Hangouts as probably they need additional parameter (notification_tag?)
    i++;
}
RemoteInput.addResultsToIntent(remoteInputs, localIntent, localBundle);
try {
    notificationWear.pendingIntent.send(MainActivity.this, 0, localIntent);
} catch (PendingIntent.CanceledException e) {
    Log.e(TAG, "replyToLastNotification error: " + e.getLocalizedMessage());
}

Our notificationWear object is a temporary container for all needed data to respond to a particular notification. Then we take each RemoteInput from the saved notification and fill it with our desired text by using the putCharSequence() method on a Bundle that we will pass back. The key here is to use getResultKey() on each RemoteInput as this is where the called app will look for the reply text in the returned Bundle. As a final step we need to populate Intent with RemoteInputs that we just filled with fake reply, to do so let’s use addResultsToIntent(inputs, intent, bundle). Now we are ready to fire PendingIntent with all that data, to trigger it let’s call pendingIntent.send(context, code,intent). This is all, we just responded to a conversation in Messenger, Telegraph, Line or any other app that uses WearableExtender with RemoteInput! This method still lacks support for Hangouts as probably they require to pass some additional parameter - our best guess is Tag of StatusBarNotification as in case of other (working) apps it is null.

Cool?

Yes it is, and it allows for great things - just take a look at Pushbullet. But the concern in our office was that the user is prompted to select which app has access to read notification and using this hack, we can respond to any notification. What is more, we not only can respond to but also cache data from notifications needed to trigger, for example, some conversation in Messenger and use it later on. Also, potentially, it allows you to send text messages without the right permissions as you “just responded to a notification” and by accident some of them are from text messages apps. For now default text messaging app in Android doesn’t include WearableExtender but probably it is just matter of time, if even Hangouts does – and they allow you to integrate with text messaging functionality.

WearableExtender

As you can see in system settings you just grant access to read notifications, but our app that is given this permission can respond to any as well.

Stay safe

What works one way, can always work the other – which means that if you are building a Wear-enabled app, other developers will be able to respond to your notifications. After looking into the RemoteInput hack, let’s think about how we can stay safe from it. The simplest way is to generate some id for each notification and allow them to be used only once - so any app using this hack won’t send more than one message to your app. Other way is to pass some argument not included in Bundle, but as a for example Tag of Notification and compare it after receiving. That is probably why we couldn’t make the hack work with Hangouts. On the other hand, if you are a user you should pay much greater attention to apps that have access to “reading notifications” as now it also allows them to interact with them. If you want a test flight or help us to make it work with Hangouts, sample app is on Github.

share


MichałSoftware Engineer

LEARN MORE

Contact us if you have any questions regarding the article or just want to chat about technology, our services, job offers and more!

POLIDEA NEWSLETTER

Sign in and expect sharp insights, recommendations, ebooks and fascinating project stories delivered to your inbox

The controller of the personal data that you are about to provide in the above form will be Polidea sp. z o.o. with its registered office in Warsaw at ul. Przeskok 2, 00-032 Warsaw, KRS number: 0000330954, tel.: 0048 795 536 436, email: hello@polidea.com (“Polidea”). We will process your personal data based on our legitimate interest and/or your consent. Providing your personal data is not obligatory, but necessary for Polidea to respond to you in relation to your question and/or request. If you gave us consent to call you on the telephone, you may revoke the consent at any time by contacting Polidea via telephone or email. You can find detailed information about the processing of your personal data in relation to the above contact form, including your rights relating to the processing, HERE.

Data controller:

The controller of your personal data is Polidea sp. z o.o. with its registered office in Warsaw at ul. Przeskok 2, 00-032 Warsaw, KRS number: 0000330954, tel.: [0048795536436], email: [hello@polidea.com] (“Polidea”)

Purpose and legal bases for processing:

 

Used abbreviations:

GDPR – Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016
on the protection of natural persons with regard to the processing of personal data and on the free movement
of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

ARES – Polish Act on Rendering Electronic Services dated 18 July 2002

TL – Polish Telecommunications Law dated 16 July 2004

1)        sending to the given email address a newsletter including information on Polidea’s new projects, products, services, organised events and/or general insights from the mobile app business world |art. 6.1 a) GDPR, art. 10.2 ARES and art. 172.1 TL (upon your consent)

Personal data:name, email address

2)       statistical, analytical and reporting purposes |art. 6. 1 f) GDPR (based on legitimate interests pursued by Polidea, consisting in analysing the way our services are used and adjusting them to our clients’ needs, as well as developing new services)

Personal data:name, email address

Withdrawal of consent:

You may withdraw your consent to process your personal data at any time.

Withdrawal of the consent is possible solely in the scope of processing performed based on the consent. Polidea is authorised to process your personal data after you withdraw your consent if it has another legal basis for the processing, for the purposes covered by that legal basis.

Categories of recipients:

Your personal data may be shared with:

1)       authorised employees and/or contractors of Polidea

2)       persons or entities providing particular services to Polidea (accounting, legal, IT, marketing and advertising services) – in the scope required for those persons or entities to provide those services to Polidea

 

Retention period:

1)       For the purpose of sending newsletter to the given email address – for as long as the relevant consent is not withdrawn

2)       For statistical, analytical and reporting purposes – for as long as the relevant consent is not withdrawn

Your rights:

 

Used abbreviation:

GDPR – Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016
on the protection of natural persons with regard to the processing of personal data and on the free movement
of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

According to GDPR, you have the following rights relating to the processing of your personal data, exercised by contacting Polidea via [e-mail, phone].

1)       to access to your personal data (art. 15 GDPR) by requesting sharing and/or sending a copy of all your personal data processed by Polidea

2)       to request rectification of inaccurate personal data
(art. 16 GDPR) by indicating the data requiring rectification

3)       to request erasure of your persona data (art. 17 GDPR); Polidea has the rights to refuse erasing the personal data in specific circumstances provided by law

4)       to request restriction of processing of your personal data (art. 18 GDPR) by indicating the data which should be restricted

5)       to move your personal data (art. 20 GDPR) by requesting preparation and transfer by Polidea of the personal data that you provided to Polidea to you or another controller in a structured, commonly used machine-readable format

6)       to object to processing your personal data conducted based on art. 6.1 e) or f) GDPR, on grounds relating to your particular situation (art. 21 GDPR)

7)       to lodge a complaint with a supervisory authority,
in particular in the EU member state of your habitual residence, place of work or place of the alleged infringement if you consider that the processing
of personal data relating to you infringes the GDPR
(art. 77.1 GDPR)

No obligation to provide data:

Providing your personal data is not obligatory, but necessary for Polidea to provide you the newsletter service

Refusal to provide the above data will result in inability to receive the newsletter service.

Profiling

In the process of providing the newsletter service, we make decisions in an automated way, including profiling, based on the data you provide.

 

“Profiling” means automated processing of personal data consisting of the use of your personal data to evaluate certain personal aspects relating to you, in particular to analyze or predict aspects concerning your personal preferences and interests.

 

The automated decisions are taken based on the analysis of clicked and viewed content. They affect the targeting of specific newsletter content to selected users registered to receive the newsletter service, based on the anticipated interests of the recipient.