Vald Filter Gateway APIs

Overview

Filter Servie is responsible for providing insert, update, upsert and search interface for Vald Filter Gateway.

Vald Filter Gateway forward user request to user-defined ingress/egress filter components allowing user to run custom logic.

Insert RPC

Insert RPC is the method to insert object through Vald Filter Gateway.

service Filter {
  rpc InsertObject(payload.v1.Insert.ObjectRequest)
      returns (payload.v1.Object.Location) {
    option (google.api.http) = {
      post : "/insert/object"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Insert.ObjectRequest

    message Insert {
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Insert.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be inserted
      configConfig*the configuration of the insert request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Insert.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector inserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.Location

    message Object {
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Location
      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is inserted.
      uuidstringthe ID of the inserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is inserted.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

StreamInsert RPC

StreamInsert RPC is the method to add new multiple object using the bidirectional streaming RPC.

By using the bidirectional streaming RPC, the insert request can be communicated in any order between client and server. Each Insert request and response are independent. It’s the recommended method to insert a large number of objects.

service Filter {
  rpc StreamInsertObject(stream payload.v1.Insert.ObjectRequest)
      returns (stream payload.v1.Object.StreamLocation) {}
}

Input

  • the scheme of payload.v1.Insert.ObjectRequest

    message Insert {
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Insert.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be inserted
      configConfig*the configuration of the insert request
      vectorizerFilter.Target*filter configurations
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Insert.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector inserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.StreamLocation

    message Object {
        message StreamLocation {
          oneof payload {
              Location location = 1;
              google.rpc.Status status = 2;
          }
        }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.StreamLocation

      fieldtypelabeldesc.
      locationObject.Locationthe information of Object.Location data.
      statusgoogle.rpc.Statusthe status of google RPC.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is inserted.
      uuidstringthe ID of the inserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is inserted.
    • google.rpc.Status

      fieldtypelabeldesc.
      codeint32status code (code list is next section)
      messagestringerror message
      detailsgoogle.protobuf.Anyrepeated(Array[any])the details error message list

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

MultiInsert RPC

MultiInsert RPC is the method to add multiple new objects in 1 request.

service Filter {
  rpc MultiInsertObject(payload.v1.Insert.MultiObjectRequest)
      returns (payload.v1.Object.Locations) {
    option (google.api.http) = {
      post : "/insert/object/multiple"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Insert.MultiObjectRequest

    message Insert {
        message MultiObjectRequest {
          repeated ObjectRequest requests = 1; 
        }
    
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Insert.MultiObjectRequest

      fieldtypelabelrequireddesc.
      requestsObjectRequestrepeated(Array[Insert.ObjectRequest])*the multiple search by binary object request content
    • Insert.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be inserted
      configConfig*the configuration of the insert request
      vectorizerFilter.Target*filter configurations
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Insert.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector inserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.Locations

    message Object {
        message Locations { repeated Location locations = 1; }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Locations

      fieldtypelabeldesc.
      locationObject.Locationrepeated(Array[Object.Location])the list of Object.Location.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is inserted.
      uuidstringthe ID of the inserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is inserted.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

Update RPC

Update RPC is the method to update a single vector.

service Filter {
  rpc UpdateObject(payload.v1.Update.ObjectRequest)
      returns (payload.v1.Object.Location) {
    option (google.api.http) = {
      post : "/update/object"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Update.ObjectRequest

    message Update {
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Update.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be updated
      configConfig*the configuration of the update request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Update.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already updated or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector updated.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.Location

    message Object {
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Location
      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is updated.
      uuidstringthe ID of the updated vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is updated.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

Stream Update RPC

StreamUpdate RPC is the method to update multiple objects using the bidirectional streaming RPC.
By using the bidirectional streaming RPC, the update request can be communicated in any order between client and server. Each Update request and response are independent. It’s the recommended method to update the large amount of objects.

service Filter {
  rpc StreamUpdateObject(stream payload.v1.Update.ObjectRequest)
      returns (stream payload.v1.Object.StreamLocation) {}
}

Input

  • the scheme of payload.v1.Update.ObjectRequest stream

    message Update {
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Update.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be updated
      configConfig*the configuration of the update request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Update.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already updated or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector updated.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.StreamLocation

    message Object {
        message StreamLocation {
          oneof payload {
              Location location = 1;
              google.rpc.Status status = 2;
          }
        }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.StreamLocation

      fieldtypelabeldesc.
      locationObject.Locationthe information of Object.Location data.
      statusgoogle.rpc.Statusthe status of google RPC.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is updated .
      uuidstringthe ID of the updated vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is updated.
    • google.rpc.Status

      fieldtypelabeldesc.
      codeint32status code (code list is next section)
      messagestringerror message
      detailsgoogle.protobuf.Anyrepeated(Array[any])the details error message list

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

MultiUpdate RPC

MultiUpdate is the method to update multiple objects in 1 request.

gRPC has the message size limitation.
Please be careful that the size of the request exceed the limit.
service Filter {
  rpc MultiUpdateObject(payload.v1.Update.MultiObjectRequest)
      returns (payload.v1.Object.Locations) {
    option (google.api.http) = {
      post : "/update/object/multiple"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Update.MultiObjectRequest

    message Update {
        message MultiObjectRequest {
          repeated ObjectRequest requests = 1;
        }
    
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Update.MultiObjectRequest

      fieldtypelabelrequireddesc.
      requestsInsert.ObjectRequestrepeated(Array[Insert.ObjectRequest])*the request list
    • Update.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be updated
      configConfig*the configuration of the update request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Update.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already updated or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector updated.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.Locations

    message Object {
        message Locations { repeated Location locations = 1; }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Locations

      fieldtypelabeldesc.
      locationObject.Locationrepeated(Array[Object.Location])the list of Object.Location.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is updated.
      uuidstringthe ID of the updated vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is updated.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

Upsert RPC

Upsert RPC is the method to update a single object and add a new single object.

service Filter {
  rpc UpsertObject(payload.v1.Upsert.ObjectRequest)
      returns (payload.v1.Object.Location) {
    option (google.api.http) = {
      post : "/upsert/object"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Upsert.ObjectRequest

    message Upsert {
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Upsert.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be upserted
      configConfig*the configuration of the upsert request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Update.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already upserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector upserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.Location

    message Object {
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Location
      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is upserted.
      uuidstringthe ID of the upserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is upserted.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

StreamUpsert RPC

Upsert RPC is the method to update a single object and add a new single object.

service Filter {
  rpc StreamUpsertObject(stream payload.v1.Upsert.ObjectRequest)
      returns (stream payload.v1.Object.StreamLocation) {}
}

Input

  • the scheme of payload.v1.Upsert.ObjectRequest stream

    message Upsert {
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Upsert.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be upserted
      configConfig*the configuration of the upsert request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Update.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already upserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector upserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.StreamLocation

    message Object {
        message StreamLocation {
          oneof payload {
              Location location = 1;
              google.rpc.Status status = 2;
          }
        }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.StreamLocation

      fieldtypelabeldesc.
      locationObject.Locationthe information of Object.Location data.
      statusgoogle.rpc.Statusthe status of google RPC.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is upserted.
      uuidstringthe ID of the upserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is upserted.
    • google.rpc.Status

      fieldtypelabeldesc.
      codeint32status code (code list is next section)
      messagestringerror message
      detailsgoogle.protobuf.Anyrepeated(Array[any])the details error message list

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

MultiUpsert RPC

MultiUpsert is the method to update existing multiple objects and add new multiple objects in 1 request.

gRPC has a message size limitation.
Please be careful that the size of the request exceeds the limit.
service Filter {
  rpc MultiUpsertObject(payload.v1.Upsert.MultiObjectRequest)
      returns (payload.v1.Object.Locations) {
    option (google.api.http) = {
      post : "/upsert/object/multiple"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Upsert.MultiObjectRequest

    message Upsert {
        message MultiObjectRequest {
          repeated ObjectRequest requests = 1;
        }
    
        message ObjectRequest {
          Object.Blob object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
        }
    }
    
    message Object {
        message Blob {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          bytes object = 2;
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Upsert.MultiObjectRequest

      fieldtypelabelrequireddesc.
      requestsUpsert.ObjectRequestrepeated(Array[Upsert.ObjectRequest])*the request list
    • Upsert.ObjectRequest

      fieldtypelabelrequireddesc.
      objectObject.Blob*the binary object to be upserted
      configConfig*the configuration of the upsert request
      vectorizerFilter.Target*filter target
    • Object.Blob

      fieldtypelabelrequireddesc.
      idstring*the object ID
      objectbytes*the binary object
    • Update.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already upserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector upserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Filter.Target

      fieldtypelabelrequireddesc.
      hoststring*the target hostname
      portport*the target port
    • Filter.Config

      fieldtypelabelrequireddesc.
      targetsFilter.Targetrepeated(Array[Filter.Target])*the filter target configuration

Output

  • the scheme of payload.v1.Object.Locations

    message Object {
        message Locations { repeated Location locations = 1; }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Locations

      fieldtypelabeldesc.
      locationObject.Locationrepeated(Array[Object.Location])the list of Object.Location.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is upserted.
      uuidstringthe ID of the upserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is upserted.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

Search RPC

Search RPC is the method to search object(s) similar to request object.

service Filter {
  rpc SearchObject(payload.v1.Search.ObjectRequest)
      returns (payload.v1.Search.Response) {
    option (google.api.http) = {
      post : "/search/object"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Search.ObjectRequest

    message Search {
        message ObjectRequest {
          bytes object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          string request_id = 1;
          uint32 num = 2 [ (validate.rules).uint32.gte = 1 ];
          float radius = 3;
          float epsilon = 4;
          int64 timeout = 5;
          Filter.Config ingress_filters = 6;
          Filter.Config egress_filters = 7;
          uint32 min_num = 8 [ (validate.rules).uint32.gte = 0 ];
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Search.ObjectRequest

      fieldtypelabelrequireddesc.
      objectbytes*the binary object to be searched
      configConfig*the configuration of the search request
      vectorizerFilter.Target*filter target
    • Search.Config

      fieldtypelabelrequireddesc.
      request_idstringunique request ID
      numuint32*the maximum number of result to be returned
      radiusfloat*the search radius
      epsilonfloat*the search coefficient (default value is 0.1)
      timeoutint64Search timeout in nanoseconds (default value is 5s)
      ingress_filtersFilter.ConfigIngress Filter configuration
      egress_filtersFilter.ConfigEgress Filter configuration
      min_numuint32the minimum number of result to be returned

Output

  • the scheme of payload.v1.Search.Response.

    message Search {
      message Response {
        string request_id = 1;
        repeated Object.Distance results = 2;
      }
    }
    
    message Object {
      message Distance {
        string id = 1;
        float distance = 2;
      }
    }
    
    • Search.Response

      fieldtypelabeldesc.
      request_idstringthe unique request ID
      resultsObject.Distancerepeated(Array[Object.Distance])search results
    • Object.Distance

      fieldtypelabeldesc.
      idstringthe vector ID
      distancefloatthe distance between result vector and request vector

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
4DEADLINE_EXCEEDED
5NOT_FOUND

StreamSearch RPC

StreamSearch RPC is the method to search vectors with multi queries(objects) using the bidirectional streaming RPC.
By using the bidirectional streaming RPC, the search request can be communicated in any order between client and server. Each Search request and response are independent.

service Filter {
  rpc StreamSearchObject(stream payload.v1.Search.ObjectRequest)
      returns (stream payload.v1.Search.StreamResponse) {}
}

Input

  • the scheme of payload.v1.Search.ObjectRequest stream

    message Search {
        message ObjectRequest {
          bytes object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          string request_id = 1;
          uint32 num = 2 [ (validate.rules).uint32.gte = 1 ];
          float radius = 3;
          float epsilon = 4;
          int64 timeout = 5;
          Filter.Config ingress_filters = 6;
          Filter.Config egress_filters = 7;
          uint32 min_num = 8 [ (validate.rules).uint32.gte = 0 ];
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Search.ObjectRequest

      fieldtypelabelrequireddesc.
      objectbytes*the binary object to be searched
      configConfig*the configuration of the search request
      vectorizerFilter.Target*filter target
    • Search.Config

      fieldtypelabelrequireddesc.
      request_idstringunique request ID
      numuint32*the maximum number of result to be returned
      radiusfloat*the search radius
      epsilonfloat*the search coefficient (default value is 0.1)
      timeoutint64Search timeout in nanoseconds (default value is 5s)
      ingress_filtersFilter.ConfigIngress Filter configuration
      egress_filtersFilter.ConfigEgress Filter configuration
      min_numuint32the minimum number of result to be returned

Output

  • the scheme of payload.v1.Search.StreamResponse.

    message Search {
      message StreamResponse {
        oneof payload {
          Response response = 1;
          google.rpc.Status status = 2;
        }
      }
    
      message Response {
        string request_id = 1;
        repeated Object.Distance results = 2;
      }
    }
    
    message Object {
      message Distance {
        string id = 1;
        float distance = 2;
      }
    }
    
    • Search.StreamResponse

      fieldtypelabeldesc.
      responseResponsethe search result response
      statusgoogle.rpc.Statusthe status of google RPC
    • Search.Response

      fieldtypelabeldesc.
      request_idstringthe unique request ID
      resultsObject.Distancerepeated(Array[Object.Distance])search results
    • Object.Distance

      fieldtypelabeldesc.
      idstringthe vector ID
      distancefloatthe distance between result vector and request vector

MultiSearch RPC

MultiSearch RPC is the method to search objects with multiple objects in 1 request.

gRPC has a message size limitation.
Please be careful that the size of the request exceeds the limit.
service Filter {
  rpc MultiSearchObject(payload.v1.Search.MultiObjectRequest)
      returns (payload.v1.Search.Responses) {
    option (google.api.http) = {
      post : "/search/object/multiple"
      body : "*"
    };
  }
}

Input

  • the scheme of payload.v1.Search.MultiObjectRequest

    message Search {
        message MultiObjectRequest { 
          repeated ObjectRequest requests = 1; 
        }
    
        message ObjectRequest {
          bytes object = 1;
          Config config = 2;
          Filter.Target vectorizer = 3;
        }
    
        message Config {
          string request_id = 1;
          uint32 num = 2 [ (validate.rules).uint32.gte = 1 ];
          float radius = 3;
          float epsilon = 4;
          int64 timeout = 5;
          Filter.Config ingress_filters = 6;
          Filter.Config egress_filters = 7;
          uint32 min_num = 8 [ (validate.rules).uint32.gte = 0 ];
        }
    }
    
    message Filter {
        message Target {
          string host = 1;
          uint32 port = 2;
        }
    
        message Config { 
          repeated Target targets = 1; 
        }
    }
    
    • Search.MultiObjectRequest

      fieldtypelabelrequireddesc.
      requestsrepeated(Array[MultiObjectRequest])*the search request list
    • Search.ObjectRequest

      fieldtypelabelrequireddesc.
      objectbytes*the binary object to be searched
      configConfig*the configuration of the search request
      vectorizerFilter.Target*filter target
    • Search.Config

      fieldtypelabelrequireddesc.
      request_idstringunique request ID
      numuint32*the maximum number of result to be returned
      radiusfloat*the search radius
      epsilonfloat*the search coefficient (default value is 0.1)
      timeoutint64Search timeout in nanoseconds (default value is 5s)
      ingress_filtersFilter.ConfigIngress Filter configuration
      egress_filtersFilter.ConfigEgress Filter configuration
      min_numuint32the minimum number of result to be returned

Output

  • the scheme of payload.v1.Search.Responses.

    message Search {
      message Responses {
        repeated Response responses = 1;
      }
    
      message Response {
        string request_id = 1;
        repeated Object.Distance results = 2;
      }
    }
    
    message Object {
      message Distance {
        string id = 1;
        float distance = 2;
      }
    }
    
    • Search.Responses

      fieldtypelabeldesc.
      responsesResponserepeated(Array[Response])the list of search results response
    • Search.Response

      fieldtypelabeldesc.
      request_idstringthe unique request ID
      resultsObject.Distancerepeated(Array[Object.Distance])search results
    • Object.Distance

      fieldtypelabeldesc.
      idstringthe vector ID
      distancefloatthe distance between result vector and request vector

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
4DEADLINE_EXCEEDED
5NOT_FOUND

See also