Machine Learning for Mobile App Developers – The Codes

Introduction

Are you an Android developer and want to get started with machine learning for mobile app developers? Well, you are at the right place.  This blog is the continuation of my previous blog where I have already given a brief about what is machine learning, why it is a next big thing or why the mobile app developers should start incorporating it in their apps.

We will be choosing Firebase ML Kit to deal with machine learning for mobile apps and here I you will get a basic introduction that how to code for one.

Ever clicked a picture or taken a screenshot of an image with so many text and faced issue in typing it or felt it is too tiring? Well, this always happens with me when I love a quote from a famous page on Instagram but I just want to modify it a bit and post it.

At the end of this blog you will be able to make an app which can uniquely recognise the text from the image and then do whatever you want with it! Isn’t this cool?

 

What you need?

  • A recent version of Android Studio (v3.0+)
  • A physical Android device
  • Basic knowledge of Android development in Java

 

Let’s get started!

Just follow these steps:

  1. Go to the Firebase Console
  2. Select Create New Project, and give a name to your project.

Connect your Android app

  1. From the overview screen of your new project,
    click Add Firebase to your Android app.
  2. Enter the codelab’s package name: com.google.firebase.codelab.mlkit.
  3. Leave the other fields blank and click Register app.

 

Add google-services.json file to your app:

After adding the package name and selecting Continue, your browser automatically downloads a configuration file that contains all the necessary Firebase metadata for your app. Copy the google-services.json file into the app directory in your project. Skip the remaining instructions for adding the Firebase SDK to your app. This has already been done for you in the starter project.

Add the dependencies:

build.gradle

dependencies {
 // ...
   implementation 'com.google.firebase:firebase-ml-vision:18.0.1'
   implementation 'com.google.firebase:firebase-ml-vision-image-label-model:17.0.2'
   implementation 'com.google.firebase:firebase-ml-vision-face-model:17.0.2'
   implementation 'com.google.firebase:firebase-ml-model-interpreter:16.2.3'
}
apply plugin: 'com.google.gms.google-services'

Once this is done, just sync your project with the gradle file.

 

Add on-device text recognition:

In this step, we will add functionality to your app to recognize text in images.

Add following lines of code in your activity to configure the text recognition detector:


       FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(mSelectedImage);
       FirebaseVisionTextRecognizer recognizer = FirebaseVision.getInstance()
               .getOnDeviceTextRecognizer();
       mTextButton.setEnabled(false);
       recognizer.processImage(image)
               .addOnSuccessListener(
                       new OnSuccessListener<FirebaseVisionText>() {
                           @Override
                           public void onSuccess(FirebaseVisionText texts) {
                               mTextButton.setEnabled(true);
                               processTextRecognitionResult(texts);
                           }
                       })
               .addOnFailureListener(
                       new OnFailureListener() {
                           @Override
                           public void onFailure(@NonNull Exception e) {
                               // Task failed with an exception
                               mTextButton.setEnabled(true);
                               e.printStackTrace();
                           }
                       });
   

 

Process the text recognition response:

Add the following code to processTextRecognitionResult in the activity class to parse the results and display them in your app.

 

  private void processTextRecognitionResult(FirebaseVisionText texts) {
       List<FirebaseVisionText.TextBlock> blocks = texts.getTextBlocks();
       if (blocks.size() == 0) {
           showToast("No text found");
           return;
       }
       mGraphicOverlay.clear();
       for (int i = 0; i < blocks.size(); i++) {
           List<FirebaseVisionText.Line> lines = blocks.get(i).getLines();
           for (int j = 0; j < lines.size(); j++) {
               List<FirebaseVisionText.Element> elements = lines.get(j).getElements();
               for (int k = 0; k < elements.size(); k++) {
                   Graphic textGraphic = new TextGraphic(mGraphicOverlay, elements.get(k));
                   mGraphicOverlay.add(textGraphic);

               }
           }
       }
   }

 

 

That’s all. Just run your app and start playing with it!

 

Conclusion

This was just a sample implementation of the FIrebase ML Kit for Android applications to detect the text from the image. Firebase also provides the same SDK for iOS devices. Currently it is in BETA stage and provides supports for common use cases such as barcode detection, text detection, face detection, landmark detection and labelling of images. But you can deploy your own models in Firebase using TensorFLow Lite and make your app work on your model. Not only this, you can even make use of Google Cloud Vision API and Neural Network API inside a single SDK of Firebase ML Kit. It’s API works on both on-device and on cloud.