WK07 – Code Development
During this week I had a major scare regarding the technical development of my project. I had previously found an Udemy course that comprised the teaching and deploying of an object detection model within a camera app using Android Studio. I successfully implemented the access to camera and gallery. The following videos show an example of how the interaction of the exercise deployed on a Virtual device of Android Studio. However, the main dependency to load a model had been deprecated and therefore it couldn’t be used. This led me to do an exhaustive search of examples online that tried to implement something similar.
Selecting ‘choose file’ which opens the phone’s gallery, and allows to select an image.
Selecting ‘Launch Camera’, which opens the phone’s camera, and allows to take pictures.
I was trying to quickly understand if the deployment of an object detection model was possible for mobile devices, and I tried to find different examples that I could quickly test. However, every time I tried to run code with a model in it, my Android Studio would error. This made me understand that I needed to build from the ground, and more importantly, understand what was available to me to build what I wanted. My search led me to learn two very important things:
- ML for mobile phones with TensorFlow Lite: it is possible to deploy object detection models with a version of TensorFlow that has been specifically designed for mobile phones. One of the biggest advantages was to know that it doesn’t need an Internet connection to work, but it also improves on :
- Latency: because there’s no round-trip to a server, the latency of the model isn’t hugely affected.
- Privacy: no data needs to leave the device
- Connectivity: Internet connection isn’t required.
- Power Consumption: on device ML isn’t as power consuming ad network connections.
2. There are two different ways in which a react native project can be deployed, and I need to be careful to understand which one might the appropriate framework for my case. The two options are:
| React Native + Android Studio | React + Expo Go |
| * Workflow: – Requires setting up native development tools separately and handling platform-specific configurations. – Provides more control over the development process. * Capabilities: – Allows direct access to native modules and APIs. – Supports both Android and iOS platforms. – Can integrate third-party libraries and custom native code. * TensorFlow Lite Compatibility: – React Native can handle TensorFlow Lite models directly. | * Workflow: – Offers a simplified development experience and rapid prototyping. – Provides cross-platform compatibility. – Eases the process of visualizing the app during development. * Capabilities: – Abstracts away native modules and provides a unified API. – Simplifies access to common device features (e.g., camera, geolocation). – Ideal for building apps quickly without dealing with native setup. * TensorFlow Lite Compatibility: – Expo Go does not directly support TensorFlow Lite. |
At this point, I had already built a project on Expo Go, and tested several others on Android Studio. Expo Go did feel like an easier setup and workflow to begin with. It also offered an app that allowed you to test your app instantly. However, one of the major drawbacks was the fact that it doesn’t support TF Lite, which I require to load a model. Therefore I chose to continue the development process with React-native.
After understanding this, I realized that I wanted to deploy a model on react-native without even understanding much of the folder structure and the different files that made up a project. Therefore, I took a quick course called ‘React Native Ecosystem and Workflow’, which was very useful in considering the following things:
For the correct folder structure, I had to do some extra research but in the end I got to understand how to organize my main screens, my components (ideally reusable), and my assets (fonts, icons, images, etc.). How content and styles are written Is very similar to HTML and CSS, which I have found useful. In the following small video, it can be seen
– How my main screens have been structured
– The implementation of bottom navigation from the react-native-paper library.
– Each icon in the menu connects to its corresponding screen
– Custom Precious Plastic font.
Next Steps
Having understood the main structure of a react-native project, and its build with Android Studio, I can proceed to action the following steps:
- Routing
- How to trigger the camera from within the app
- How to add a model within React native
- How to capture and record the object-detection results.
- Start thinking how I can deploy this app to be tested on a device, instead of the simulator.