Vald Insert APIs

Overview

Insert Service is responsible for inserting new vectors into the vald-agent.

service Insert {
    rpc Insert(payload.v1.Insert.Request) returns (payload.v1.Object.Location) {}

    rpc StreamInsert(stream payload.v1.Insert.Request) returns (stream payload.v1.Object.Location) {}

    rpc MultiInsert(payload.v1.Insert.MultiRequest) returns (payload.v1.Object.Locations) {}
}

Insert RPC

Inset RPC is the method to add a new single vector.

Input

  • the scheme of payload.v1.Insert.Request

    message Insert {
        message Request {
            Object.Vector vector = 1 [ (validate.rules).repeated .min_items = 2 ];
            Config config = 2;
        }
    
        message Config {
            bool skip_strict_exist_check = 1;
            Filter.Config filters = 2;
            int64 timestamp = 3;
        }
    }
    
    message Object {
        message Vector {
            string id = 1 [ (validate.rules).string.min_len = 1 ];
            repeated float vector = 2 [ (validate.rules).repeated .min_items = 2 ];
        }
    }
    
    • Insert.Request

      fieldtypelabelrequireddesc.
      vectorObject.Vector*the information of vector
      configConfig*the configuration of the insert request
    • Insert.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector inserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Object.Vector

      fieldtypelabelrequireddesc.
      idstring*the ID of a vector. ID should consist of 1 or more characters.
      vectorfloatrepeated(Array[float])*the vector data. its dimension is between 2 and 65,536.

Output

  • the scheme of payload.v1.Object.Location

    message Object {
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Location
      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is inserted.
      uuidstringthe ID of the inserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is inserted.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

StreamInsert RPC

StreamInsert RPC is the method to add new multiple vectors using the bidirectional streaming RPC.
By using the bidirectional streaming RPC, the insert request can be communicated in any order between client and server. Each Insert request and response are independent. It’s the recommended method to insert a large number of vectors.

Input

  • the scheme of payload.v1.Insert.Request stream

    message Insert {
        message Request {
            Object.Vector vector = 1 [ (validate.rules).repeated .min_items = 2 ];
            Config config = 2;
        }
        message Config {
            bool skip_strict_exist_check = 1;
            Filter.Config filters = 2;
            int64 timestamp = 3;
        }
    }
    
    message Object {
        message Vector {
            string id = 1 [ (validate.rules).string.min_len = 1 ];
            repeated float vector = 2 [ (validate.rules).repeated .min_items = 2 ];
        }
    }
    
    • Insert.Request

      fieldtypelabelrequireddesc.
      vectorObject.Vector*the information of vector
      configConfig*the configuration of the insert request
    • Insert.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector inserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Object.Vector

      fieldtypelabelrequireddesc.
      idstring*the ID of the vector. ID should consist of 1 or more characters.
      vectorfloatrepeated(Array[float])*the vector data. its dimension is between 2 and 65,536.

Output

  • the scheme of payload.v1.Object.StreamLocation

    message Object {
        message StreamLocation {
          oneof payload {
              Location location = 1;
              google.rpc.Status status = 2;
          }
        }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.StreamLocation

      fieldtypelabeldesc.
      locationObject.Locationthe information of Object.Location data.
      statusgoogle.rpc.Statusthe status of google RPC.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is inserted.
      uuidstringthe ID of the inserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is inserted.
    • google.rpc.Status

      fieldtypelabeldesc.
      codeint32status code (code list is next section)
      messagestringerror message
      detailsgoogle.protobuf.Anyrepeated(Array[any])the details error message list

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

MultiInsert RPC

MultiInsert RPC is the method to add multiple new vectors in 1 request.

gRPC has a message size limitation.
Please be careful that the size of the request exceeds the limit.

Input

  • the scheme of payload.v1.Insert.MultiRequest

    message Insert {
        message MultiRequest { repeated Request requests = 1; }
    
        message Request {
            Object.Vector vector = 1 [ (validate.rules).repeated .min_items = 2 ];
            Config config = 2;
        }
    
        message Config {
            bool skip_strict_exist_check = 1;
            Filter.Config filters = 2;
            int64 timestamp = 3;
        }
    }
    
    message Object {
        message Vector {
            string id = 1 [ (validate.rules).string.min_len = 1 ];
            repeated float vector = 2 [ (validate.rules).repeated .min_items = 2 ];
        }
    }
    
    • Insert.MultiRequest

      fieldtypelabelrequireddesc.
      requestsInsert.Requestrepeated(Array[Insert.Request])*the request list
    • Insert.Request

      fieldtypelabelrequireddesc.
      vectorObject.Vector*the information of vector
      configConfig*the configuration of the insert request
    • Insert.Config

      fieldtypelabelrequireddesc.
      skip_strict_exist_checkboolcheck the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestampint64the timestamp of the vector inserted.
      if it is N/A, the current time will be used.
      filtersFilter.Configconfiguration for filter
    • Object.Vector

      fieldtypelabelrequireddesc.
      idstring*the ID of a vector. ID should consist of 1 or more characters.
      vectorfloatrepeated(Array[float])*the vector data. its dimension is between 2 and 65,536.

Output

  • the scheme of payload.v1.Object.Locations.

    message Object {
        message Locations { repeated Location locations = 1; }
    
        message Location {
          string name = 1;
          string uuid = 2;
          repeated string ips = 3;
        }
    }
    
    • Object.Locations

      fieldtypelabeldesc.
      locationObject.Locationrepeated(Array[Object.Location])the list of Object.Location.
    • Object.Location

      fieldtypelabeldesc.
      namestringthe name of vald agent pod where the request vector is inserted.
      uuidstringthe ID of the inserted vector. It is the same as an Object.Vector.
      ipsstringrepeated(Array[string])the IP list of vald-agent pods where the request vector is inserted.

Status Code

codedesc.
0OK
3INVALID_ARGUMENT
6ALREADY_EXISTS
13INTERNAL

See also