Firebase Genkit - AI Application by Javascript/Typescript

Medium: Firebase Genkit - AI Application by Javascript/Typescript

Google recently launched the Firebase Genkit as an open source framework to build, deploy, and monitor AI applications.

Overall, you guys are constantly launching AI-related technology, and you see it's just like an arms race, everyone wants their products to be better than others from OpenAI with the GTP-4o that was just launched, and Google with the Gemini that's parallel to the Build with AI series of events to learn technology with Vertex AI...

Back to the main content of this article, Genkit is used to create custom content creation applications, semantic searches, input processing, and answering questions based on the data you set up, automated decision-making, command coordination, and more. You can find out more on Firebase's home page about Genkit attached here.

Genkit currently supports JavaScript/TypeScript development (Node.js).

Key Features

From the information page of Firebase Genkit there are highlights 10 features that are considered as key features so in this article let's find out what those 10 features are!

1. Many models, one interface

That means you can use this feature to integrate image recognition models, text translation models, and text creation models into the same application.

2. Structured output

This means that the data returned from Genkit will be a structured output, so returning it will make it easier for you to handle the data. For example, when using an image recognition model, the results you can get are information about the objects detected in the image, including location, size, etc.

import { generate } from "@genkit-ai/ai";
import { geminiPro } from "@genkit-ai/vertexai";
import { z } from "zod";

const LocationSchema = z.object({
  name: z.string().describe('Location name'),
  address: z.string().describe('Address'),
  reviews: z.array(z.object({
    name: z.string(),
    goodReview: z.number().describe('Number of good review'),
    badReview: z.number().describe('Number of bad review'),
  })).describe('3 properties that model can be response!')
});

const createLocation = defineFlow({
    name: "createLocation",
    inputSchema: z.string(),
    outputSchema: LocationSchema,
  },
  (habitat) => {
    const result = await generate({
      model: geminiPro,
      prompt: `You were created successfully for the location: ${habitat}.`,
      output: {schema: LocationSchema}
    });
    // strongly typed and ready to go
    return result.output();
  }
)

console.log(await createCreature("a developer conference"));

3. Multimodal, multimedia

Genkit supports working with different types of input and output data, such as text, images, audio and video. Simply understand, this feature can be applied to create an image object recognition app, translate text into speech, and create music based on text.

import { imagen2, geminiProVision } from '@genkit-ai/vertexai';
import { generate } from '@genkit-ai/ai';

const imageResult = await generate({
  model: imagen2,
  prompt: 'Create image include time, location and history information of that location',
});
const generatedImage = imageResult.media();

const descriptionResult = await generate({
  model: geminiProVision,
  prompt: [
    {
      text: 'What is the time, location and history information the image express?',
    },
    { media: generatedImage },
  ],
});
console.log(descriptionResult.text());

4. LLM

Genkit provides special tools for the LLM, such as the ability to access memory and context to generate better results. For example, when you build a chatbot, you can apply this feature to improve accuracy and fluidity for outputs based on the LLM you provide in the application.

import { generate, defineTool } from "@genkit-ai/ai";
import { geminiPro } from "@genkit-ai/vertexai";
import { z } from "zod";

const createReminder = defineTool({
  name: "createReminder",
  description: "Create Reminder",
  inputSchema: z.object({
    time: z.string().describe('ISO timestamp string, e.g. 2024-04-03T12:23:00Z'),
    reminder: z.string().describe('The content of reminder'),
  }),
  outputSchema: z.number().describe('ID of reminder'),
  (reminder) => db.reminders.create(reminder)
});

const searchNotes = defineTool({
  name: "searchNotes",
  description: "Use this to find the person or the note content",
  inputSchema: z.string().describe('the search query'),
  outputSchema: z.object({notes: z.array(NoteSchema)}),
  (query) => db.notes.search(query)
});

const result = await generate({
  model: geminiPro,
  tools: [createReminder, searchNotes],
  prompt: `
  You're a note assistant. Use the tools available, try to answer the question.
  If you create a reminder, describe your reminder in writing as a feedback.
  
  Question: I've recorded my appointment with my friend - can you put a time reminder?
  `
});
console.log(result.text());

5. Manage Prompt with Dotprompt

Dotprompt is an integrated tool in Genkit that can easily create, manage and test prompt for machine learning models. In the document on the Firebase page, it explains that Dotprompt is a reminder file format that allows you to put it all into a single file for easier organization and checking.

Note: Prompt is a text that is used to guide the model on the task you want it to perform.

The key is to use Dotprompt to create different prompt for text-generating patterns such as "Write a status sentence when traveling at sea to post a photo on Facebook" or "Writ an article to promote the Google IO 2024 event in Ho Chi Minh City on June 16, 2024".

---
model: vertexai/gemini-1.0-pro
config:
  temperature: 0.9
input:
  schema:
    properties:
      location: {type: string}
      time: {type: string}
      name: {type: string}
    required: [location]
  default:
    location: Ho Chi Minh city
---

Google IO 2024 is Google's upcoming highlight event will come to {{location}}.  

For special guests {{name}} {{/if}}{{#if time}} occurred at the time {{time}}{ {/ if}}.

6. Run flows locally

Genkit allows you to run local application flows on your computer without deploying applications to the Cloud, making it easier for you to check and fix bugs during development.

7. Inspect traces

Debugging complex, multi-step workflows with AI can be difficult due to randomness and hidden processes. Genkit provides tools to track your application. Tracks are detailed records of how your application interacts with Firebase services and machine learning models. This helps you debug your application and improve performance.

If you're building a chatbot, you can see your chatbot processing user requests and identifying any accuracy or performance issues.

Note: It's hard to imagine the interpretation in words, so for a better understanding, you can see how it works based on the descriptive image [here] (https://firebase.google.com/docs/genkit#7_inspect_traces)

8. Open & extensible

Simply put, Genkit is designed to enable scaling and customization so it can add new functions and integrate third-party services. For example, it's possible to use third-party voice recognition services to develop applications.

Or you can use Genkit and then develop it into a separate plugin and then upload it to npm.

9. Built for production

Easily expand your streams to any platform, Genkit is fully equipped with OpenTelemetry and customizes metadata to monitor production.

Additionally, there are official Google Cloud and Firebase plugins that allow you to export Google Cloud activity data and integrate firebase services such as Cloud for FireBase, Firebaze Authentication, Application Testing, and Firestore.

To illustrate this feature, imagine you're developing an app like Google Trans. This application will have to handle a huge volume of translation requests and in the long run this application will emerge and there will be a huge number of users you need to respond to how to keep your application running stable with such a large volume of users instead of thinking and solving such problems Genkit will take care of this.

10. Authorization & security handling

When building any public application, it is important to protect the data stored in your system. When it comes to LLM, there is a need for further in-depth to ensure that the model only accesses the necessary data, the tool call commands are defined to the appropriate range for the user calling LLM and the streams are only called by the validated client applications.

Genkit supports a variety of authentication methods, such as Firebase and OAuth, combined with authorization policies and contexts management mechanisms.

The Genkit app creates an application that users need to log in to use and decentralize content that can be used by that user.

import { defineFlow, runFlow } from '@genkit-ai/flow';

export const selfSummaryFlow = defineFlow(
  {
    name: 'selfSummaryFlow',
    inputSchema: z.object({uid: z.string()}),
    outputSchema: z.string(),
    authPolicy: (auth, input) => {
      if (!auth) {
        throw new Error('Authorization required.');
      }
      if (input.uid !== auth.uid) {
        throw new Error('You may only summarize your own profile data.');
      }
    }
  },
  async (input) => { ... });

Review

Genkit is a new technology introduced recently, so the above article is based on the main information on the page [Firebase documents] (https://firebase.google.com/docs/genkit) and edited in my own way 😎, so if there are any errors, please contribute to the comments below.

I don't have time to study Genkit in depth either, and if you have any articles or information about the Firebase Genkit, please do not hesitate to share it below!

References: firebase.google.com/docs/genkit

I am always looking for feedback on my writing, so please let me know what you think. ❤️