Vald Update APIs

Overview

Update Service updates to new vector from inserted vector in the vald-agent components.

service Update {
    rpc Update(payload.v1.Update.Request) returns (payload.v1.Object.Location) {}

    rpc StreamUpdate(stream payload.v1.Update.Request) returns (stream payload.v1.Object.Location) {}

    rpc MultiUpdate(payload.v1.Update.MultiRequest) returns (stream payload.v1.Object.Locations) {}
}

Update RPC

Update RPC is the method to update a single vector.

Input

• the scheme of payload.v1.Update.Request

  message Update {
    message Request {
        Object.Vector vector = 1 [ (validate.rules).repeated .min_items = 2 ];
        Config config = 2;
    }
  
    message Config {
        bool skip_strict_exist_check = 1;
        Filter.Config filters = 2;
        int64 timestamp = 3;
    }
  }
  
  message Object {
      message Vector {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          repeated float vector = 2 [ (validate.rules).repeated.min_items = 2 ];
      }
  }
  

  • Update.Request

    field	type	label	required	description
    vector	Object.Vector		*	The information of vector.
    config	Config		*	The configuration of the update request.

  • Update.Config

    field	type	label	required	description
    skip_strict_exist_check	bool			Check whether the same vector is already inserted or not. The ID should be unique if the value is true.
    timestamp	int64			The timestamp of the vector inserted. If it is N/A, the current time will be used.
    filters	Filter.Config			Configuration for filter.
    disable_balanced_update	bool			A flag to disable balanced update (split remove -> insert operation) during update operation.

  • Object.Vector

    field	type	label	required	description
    id	string		*	The ID of a vector. ID should consist of 1 or more characters.
    vector	float	repeated(Array[float])	*	The vector data. Its dimension is between 2 and 65,536.

Output

• the scheme of payload.v1.Object.Location

  message Object {
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is updated.
    uuid	string		The ID of the updated vector. It is the same as an Object.Vector.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is updated.

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

StreamUpdate RPC

StreamUpdate RPC is the method to update multiple vectors using the bidirectional streaming RPC.
Using the bidirectional streaming RPC, the update request can be communicated in any order between client and server. Each Update request and response are independent. It’s the recommended method to update the large amount of vectors.

Input

• the scheme of payload.v1.Update.Request stream

  message Update {
      message Request {
          Object.Vector vector = 1 [ (validate.rules).repeated .min_items = 2 ];
          Config config = 2;
      }
      message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
      }
  }
  
  message Object {
      message Vector {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          repeated float vector = 2 [ (validate.rules).repeated .min_items = 2 ];
      }
  }
  

  • Update.Request

    field	type	label	required	description
    vector	Object.Vector		*	The information of vector.
    config	Config		*	The configuration of the update request.

  • Update.Config

    field	type	label	required	description
    skip_strict_exist_check	bool			Check whether the same vector is already inserted or not. The ID should be unique if the value is true.
    timestamp	int64			The timestamp of the vector inserted. If it is N/A, the current time will be used.
    filters	Filter.Config			Configuration for filter.
    disable_balanced_update	bool			A flag to disable balanced update (split remove -> insert operation) during update operation.

  • Object.Vector

    field	type	label	required	description
    id	string		*	The ID of a vector. ID should consist of 1 or more characters.
    vector	float	repeated(Array[float])	*	The vector data. Its dimension is between 2 and 65,536.

Output

• the scheme of payload.v1.Object.StreamLocation

  message Object {
      message StreamLocation {
        oneof payload {
            Location location = 1;
            google.rpc.Status status = 2;
        }
      }
  
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.StreamLocation

    field	type	label	description
    location	Object.Location		The information of Object.Location data.
    status	google.rpc.Status		The status of Google RPC.

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is updated.
    uuid	string		The ID of the updated vector. It is the same as an Object.Vector.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is updated.

  • google.rpc.Status

    field	type	label	description
    code	int32		Status code (code list is next section)
    message	string		Error message
    details	google.protobuf.Any	repeated(Array[any])	The details error message list

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

MultiUpdate RPC

MultiUpdate is the method to update multiple vectors in 1 request.

Input

• the scheme of payload.v1.Update.MultiRequest

  message Update {
      message MultiRequest { repeated Request requests = 1; }
  
      message Request {
          Object.Vector vector = 1 [ (validate.rules).repeated .min_items = 2 ];
          Config config = 2;
      }
  
      message Config {
          bool skip_strict_exist_check = 1;
          Filter.Config filters = 2;
          int64 timestamp = 3;
      }
  }
  
  message Object {
      message Vector {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
          repeated float vector = 2 [ (validate.rules).repeated .min_items = 2 ];
      }
  }
  

  • Update.MultiRequest

    field	type	label	required	description
    requests	Insert.Request	repeated(Array[Insert.Request])	*	The request list.

  • Update.Request

    field	type	label	required	description
    vector	Object.Vector		*	The information of vector.
    config	Config		*	The configuration of the update request.

  • Update.Config

    field	type	label	required	description
    skip_strict_exist_check	bool			Check whether the same vector is already inserted or not. The ID should be unique if the value is true.
    timestamp	int64			The timestamp of the vector inserted. If it is N/A, the current time will be used.
    filters	Filter.Config			Configuration for filter.
    disable_balanced_update	bool			A flag to disable balanced update (split remove -> insert operation) during update operation.

  • Object.Vector

    field	type	label	required	description
    id	string		*	The ID of a vector. ID should consist of 1 or more characters.
    vector	float	repeated(Array[float])	*	The vector data. Its dimension is between 2 and 65,536.

Output

• the scheme of payload.v1.Object.Locations.

  message Object {
      message Locations { repeated Location locations = 1; }
  
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.Locations

    field	type	label	description
    location	Object.Location	repeated(Array[Object.Location])	The list of Object.Location.

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is updated.
    uuid	string		The ID of the updated vector. It is the same as an Object.Vector.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is updated.

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

See also