Capacitor Plugin for Text Detection Part 1 : Create Plugin

Vennela Kodali April 1, 2020

Introduction

In this series of blog posts, I'm going to walk through building a Capacitor Plugin, specifically a Text Detection plugin using CoreML and MLKit. Capacitor plugins are handy when you're writing a Hybrid App with Ionic or a PWA but want to leverage native functionalities of the device as well.

This is part 1/6 of the "Capacitor Plugin for Text Detection" series. In this post, I'm going to walk through creating a new plugin and a sample app to work with it.

Create plugin

To create a plugin, run npx @capacitor/cli plugin:generate from the command line. This will start a wizard prompting you to enter information about the plugin. Here is an example:

> npx @capacitor/cli plugin:generate
npx: installed 57 in 4.19s
✏️  Creating new Capacitor plugin
? Plugin NPM name (kebab-case): cap-ml
? Plugin id (domain-style syntax. ex: com.example.plugin) com.bendyworks.CapML
? Plugin class name (ex: AwesomePlugin) CapML
? description: Capacitor ML Plugins
? git repository: cap-ml
? author: Vennela Kodali
? license: Apache 2
? package.json will be created, do you want to continue? Yes

In your newly created plugin directory, you'll notice ios, android, and src (for the web part of the plugin) directories.

Create Sample Project

Let's create a sample project to help us test the plugin we just created.

From the root of the plugin directory (cap-ml in my case), start a new Ionic Project with ionic start ImageReader. In the wizard that follows, choose Angular or React based on your comfort level. I chose Angular, with a blank template, to keep the app simple.

Adding Capacitor

Go into the sample app and add capacitor integration to the project with:

cd ImageReader
ionic integrations enable capacitor

Then, initialize the app with npx cap init [appName] [appId]

npx cap init ImageReader com.bendyworks.CapML.ImageReader

This will create a capacitor.config.json with the provided app name and app id. For our ios plugin, we'll use CoreML's Vision Framework which supports text detection on iOS 13.0 or higher. So, open capacitor.config.json add and entry for iOS:

 "ios": { "minVersion": "13.0"}

Build the app

Build the project with npm i && npm run build. This creates the www folder that Capacitor has been automatically configured to use as the webDir in capacitor.config.json.

Add Platforms

Add the iOS platform to your sample project with npx cap add ios
Add the Android platform with npx cap add android

From capacitor's website,

These are entirely separate native project artifacts that should be considered part of your Ionic app (i.e., check them into source control, edit them in their own IDEs, etc.).

Open Platforms

Open the native projects we created in the previous step.

To open the iOS project (ImageReader/ios) in XCode : npx cap open ios
To open the android project (ImageReader/android) in AndroidStudio : npx cap open android

Install the Plugin

Install the Plugin from ImageReader's root directory npm install file:../cap-ml

Open ImageReader/src/app/home/home.page.ts and import the plugin

import { Plugins } from '@capacitor/core';
const { CapML } = Plugins;

Out of the box plugin came with an 'echo' implementation that can be called here. For now, let's keep it simple and call echo from ngOnInit, to see the plugin in work.

export class HomePage implements OnInit {
  async ngOnInit() {
    const result = await CapML.echo({value: 'hello'});
    console.log(result)
  }
}

Every time you make changes in your app (ImageReader/src/)
- make sure to build the app npm run build
- copy over the changes to ios and android projects: npx cap copy or npx cap sync

Running the app

To run the app on the web (this uses the web plugin in the src directory):

ionic serve

To run the app on iOS (this uses the ios plugin):

npx cap open ios
Build and run the app

To run the app on Android (this uses the android plugin):

npx cap open android
Register the plugin
- Open app/java/package-name/MainActivity.java, in my case, app/java/com.bendyworks.CapML.ImageReader/MainActivity.java. In MainActivity.java
- import the plugin class - import com.bendyworks.capML.CapML;
- Inside onCreate -> init, register the plugin - add(CapML.class)
Build and run the app