Image search

Image search uses an image as a search input to perform vector similarity search.

Additional information

Configure image search

To use images as search inputs, configure an image vectorizer integration for your collection. See the model provider integrations page for a list of available integrations.

By local image path

Use the Near Image operator to execute image search.
If your query image is stored in a file, you can use the client library to search by its filename.

Python Client v4

from pathlib import Path

dogs = client.collections.get("Dog")
response = dogs.query.near_image(
    near_image=Path("./images/search-image.jpg"),  # Provide a `Path` object
    return_properties=["breed"],
    limit=1,
    # targetVector: "vector_name" # required when using multiple named vectors
)

print(response.objects[0])

client.close()

Python Client v3

response = (
    client.query
    .get("Dog", "breed")
    .with_near_image({"image": "image.jpg"})  # default `encode=True` reads & encodes the file
    .with_limit(1)
    .do()
)

JS/TS Client v3

Not available yet. Vote for the feature request. DYI code below.

const myCollection = client.collections.get('Dog');

// Read the file
const image = await fs.promises.readFile('image.jpg');

// Query based on the image content
const result = await myCollection.query.nearImage(image, {
  returnProperties: ['breed'],
  limit: 1,
  // targetVector: 'vector_name' // required when using multiple named vectors
})

console.log(JSON.stringify(result.objects, null, 2));

JS/TS Client v2

Not available yet. Vote for the feature request. DYI code below.

// Read the file into a base-64 encoded string
const contentsBase64 = await fs.promises.readFile('image.jpg', { encoding: 'base64' });

// Query based on base64-encoded image
result = await client.graphql
  .get()
  .withClassName('Dog')
  .withNearImage({
    image: contentsBase64,
  })
  .withLimit(1)
  .withFields('breed')
  .do();

console.log(JSON.stringify(result, null, 2));

Go

response, err := client.GraphQL().Get().
  WithClassName("Dog").
  WithFields(graphql.Field{Name: "breed"}).
  WithNearImage((&graphql.NearImageArgumentBuilder{}).WithImage("image.jpg")).
  WithLimit(1).
  Do(ctx)

Example response

{
  "data": {
    "Get": {
      "Dog": [
        {
          "breed": "Corgi"
        }
      ]
    }
  }
}

client.close()

By the base64 representation

You can search by a base64 representation of an image:

Python Client v4

base64_string="SOME_BASE_64_REPRESENTATION"

# Get the collection containing images
dogs = client.collections.get("Dog")

# Perform query
response = dogs.query.near_image(
    near_image=base64_string,
    return_properties=["breed"],
    limit=1,
    # targetVector: "vector_name" # required when using multiple named vectors
)

print(response.objects[0])

client.close()

Python Client v3

base64_string="SOME_BASE_64_REPRESENTATION"

# Perform query
response = (
    client.query
    .get("Dog", "breed")
    .with_near_image(
        {"image": base64_string},
        encode=False  # False because the image is already base64-encoded
    )
    .with_limit(1)
    .do()
)

print(json.dumps(response, indent=2))

JS/TS Client v3

// Perform query
const myCollection = client.collections.get('Dog');

const base64String = 'SOME_BASE_64_REPRESENTATION';

const result = await myCollection.query.nearImage(base64String, {
  returnProperties: ['breed'],
  limit: 1,
  // targetVector: 'vector_name' // required when using multiple named vectors
})

console.log(JSON.stringify(result.objects, null, 2));

JS/TS Client v2

const base64String = 'SOME_BASE_64_REPRESENTATION';

// Perform query
let result = await client.graphql
  .get()
  .withClassName('Dog')
  .withNearImage({
    image: base64String,
  })
  .withLimit(1)
  .withFields('breed')
  .do();

console.log(JSON.stringify(result, null, 2));

Go

response, err := client.GraphQL().Get().
  WithClassName("Dog").
  WithFields(graphql.Field{Name: "breed"}).
  WithNearImage((&graphql.NearImageArgumentBuilder{}).WithImage(base64String)).
  WithLimit(1).
  Do(ctx)

Example response

{
  "data": {
    "Get": {
      "Dog": [
        {
          "breed": "Corgi"
        }
      ]
    }
  }
}

client.close()

Create a base64 representation of an online image.

You can create a base64 representation of an online image, and use it as input for similarity search as shown above.

Python

import base64, requests

def url_to_base64(url):
    image_response = requests.get(url)
    content = image_response.content
    return base64.b64encode(content).decode("utf-8")

base64_img = url_to_base64("https://upload.wikimedia.org/wikipedia/commons/thumb/1/14/Deutsches_Museum_Portrait_4.jpg/500px-Deutsches_Museum_Portrait_4.jpg")

client.close()

JS/TS

import { readFileSync } from 'fs'

const urlToBase64 = async (imageUrl) => {
  const response = await fetch(imageUrl);
  const content = await response.buffer();
  return content.toString('base64');
}

urlToBase64('https://upload.wikimedia.org/wikipedia/commons/thumb/1/14/Deutsches_Museum_Portrait_4.jpg/500px-Deutsches_Museum_Portrait_4.jpg')
  .then(base64 => { console.log(base64) });

Go

resp, err := http.Get(url)
if err != nil {
  return "", err
}
defer resp.Body.Close()

content, err := ioutil.ReadAll(resp.Body)
if err != nil {
  return "", err
}

base64string := base64.StdEncoding.EncodeToString(content)

Combination with other operators

A Near Image search can be combined with any other operators (like filter, limit, etc.), just as other similarity search operators.

See the similarity search page for more details.

Related pages

• Connect to Weaviate

Questions and feedback

If you have any questions or feedback, let us know in the user forum.

• Technical questions
• Documentation feedback