Firebase GenKit with Gemma using Ollama (English)

in Firebase / Google Cloud Platform (GCP) on Firebase, Gcp Last modified at:
Firebase GenKit with Gemma using Ollama (English)

Firebase project that uses the Gen AI Kit with Gemma using Ollama

  1. Introduction
  2. Setup
    1. Open Genkit UI
    2. Run the Firebase emulator
    3. Run Gemma with Ollama
  3. Code explanation
  4. Invoke the function locally
  5. Deploy
  6. Resources
  7. Conclusion

Introduction

This is a simple example of a Firebase function that uses Genkit and Ollama to translate any test to Spanish.

This project uses the following technologies:

  1. Firebase Functions
  2. Firebase Genkit
  3. Ollama

This project uses the following Node.js Packages:

  1. genkitx-ollama: Genkit Ollama plugin to be able to use Ollama in Genkit
  2. genkit: Genkit AI Core SDK

Setup

  1. Clone this repository: GitHub repository.
  2. Run npm install to install the dependencies in the functions folder
  3. Run firebase login to login to your Firebase account
  4. Install genkit-cli by running npm install -g genkit

This repo is supposed to be used with NodeJS version 20.

Open Genkit UI

Go to the functions folder and run npm run genkit:start to open the Genkit UI. The UI will be available at http://localhost:4000.

Full-width image Firebase Genkit UI

Run the Firebase emulator

To run the function locally, run firebase emulators:start --inspect-functions.

The emulator will be available at http://localhost:4001

Run Gemma with Ollama

You will need to install Ollama by running brew install ollama and then run ollama run gemma to start the Ollama server running the Gemma LLM.

Code explanation

The code is in the functions/index.ts file. The function is called translatorFlow and it uses the Genkit SDK to translate any given text to Spanish.

First, we have to configure the Genkit SDK with the Ollama plugin:

const ai = genkit({
  plugins: [
    ollama({
      models: [{ name: 'gemma' }],
      serverAddress: 'http://127.0.0.1:11434', // default ollama local address
    }),
  ]
});
logger.setLogLevel('debug');

Then, we define the function, in the Gen AI Kit they call it Flows. A Flow is a function with some additional characteristics: they are strongly typed, streamable, locally and remotely callable, and fully observable. Firebase Genkit provides CLI and Developer UI tooling for working with flows (running, debugging, etc):

export const translatorFlow = ai.defineFlow(
  {
    name: "translatorFlow",
    inputSchema: z.object({ text: z.string() }),
    outputSchema: z.string(),
  },
  async (toTranslate) => {
    const prompt =
      `Translate this ${toTranslate.text} to Spanish. Autodetect the language.`;

    const llmResponse = await ai.generate({
      model: 'ollama/gemma',
      prompt: prompt,
      config: {
        temperature: 1,
      },
    });

    return llmResponse.text;
  }
);

export const translatedFunction = onCallGenkit({
  authPolicy: () => true, // Allow all users to call this function. Not recommended for production.
}, translatorFlow);

As we saw above, we use Zod to define the input and output schema of the function. We also use the generate function from the Genkit SDK to generate the translation.

Invoke the function locally

Now you can invoke the function by running genkit flow:run translatorFlow '{"text":"hi"}' in the terminal.

You can also make a curl command by running curl -X GET -H "Content-Type: application/json" -d '{"data": { "text": "hi" }}' http://127.0.0.1:5001/<firebase-project>/<region>/translatedFunction in the terminal.

For example:

> curl -X GET -H "Content-Type: application/json" -d '{"data": { "text": "hi" }}' http://127.0.0.1:5001/action-helloworld/us-central1/translatedFunction
{"result":"Hola\n\nThe translation of \"hi\" to Spanish is \"Hola\"."}

You can also use Postman or any other tool to make a GET request to the function:

Full-width image Postman Request

Deploy

To deploy the function, run firebase deploy --only functions. You will need to change the ollama URL in the function to the URL of the Ollama server.

Resources

Conclusion

As you can see, it is very easy to use Genkit and Ollama in Firebase Functions. You can use this example as a starting point to create your own functions using Genkit and Ollama.

You can find the full code of this example in the GitHub repository

Happy coding!

Made with ❤ by Xavier Portilla Edo © 2021. All rights reserved.