Vald Remove APIs

Overview

Remove Service is responsible for removing vectors indexed in the vald-agent.

service Remove {

  rpc Remove(payload.v1.Remove.Request) returns (payload.v1.Object.Location) {}

  rpc RemoveByTimestamp(payload.v1.Remove.TimestampRequest) returns (payload.v1.Object.Locations) {}

  rpc StreamRemove(stream payload.v1.Remove.Request)
      returns (stream payload.v1.Object.StreamLocation) {}

  rpc MultiRemove(payload.v1.Remove.MultiRequest)
      returns (payload.v1.Object.Locations) {}
}

Remove RPC

Remove RPC is the method to remove a single vector.

Input

• the scheme of payload.v1.Remove.Request

  message Remove {
      message Request {
          Object.ID id = 1;
          Config config = 2;
      }
  
      message Config {
          bool skip_strict_exist_check = 1;
          int64 timestamp = 3;
      }
  }
  
  message Object {
      message ID {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
      }
  }
  

  • Remove.Request

    field	type	label	required	description
    id	Object.ID		*	The ID of vector.
    config	Config		*	The configuration of the remove request.

  • Remove.Config

    field	type	label	required	description
    skip_strict_exist_check	bool			Check whether the same vector is already inserted or not. The ID should be unique if the value is true.
    timestamp	int64			The timestamp of the vector removed. If it is N/A, the current time will be used.

  • Object.ID

    field	type	label	required	description
    id	string		*	The ID of a vector. ID should consist of 1 or more characters.

Output

• the scheme of payload.v1.Object.Location

  message Object {
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is removed.
    uuid	string		The ID of the removed vector. It is the same as an Object.ID.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is removed.

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

RemoveByTimestamp RPC

RemoveByTimestamp RPC is the method to remove vectors based on timestamp.

Input

• the scheme of payload.v1.Remove.TimestampRequest

  message Remove {
      message TimestampRequest {
          repeated Timestamp timestamps = 1;
      }
  
      message Timestamp {
          enum Operator {
              Eq = 0;
              Ne = 1;
              Ge = 2;
              Gt = 3;
              Le = 4;
              Lt = 5;
          }
          int64 timestamp = 1;
          Operator operator = 2;
      }
  }
  
  message Object {
      message ID {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
      }
  }
  

  • Remove.TimestampRequest

    field	type	label	required	description
    timestamps	Remove.Timestamp	repeated(Array[Remove.Timestamp])	*	The timestamp comparison list. If more than one is specified, the AND search is applied.

  • Remove.Timestamp

    field	type	label	required	description
    timestamp	int64		*	The timestamp.
    operator	Remove.Timestamp.Operator			The conditionl operator. (default value is Eq).

  • Remove.Timestamp.Operator

    value	description
    Eq	Equal.
    Ne	Not Equal.
    Ge	Greater than or Equal.
    Gt	Greater than.
    Le	Less than or Equal.
    Lt	Less than.

  In the TimestampRequest message, the 'timestamps' field is repeated, allowing the inclusion of multiple Timestamp.
  When multiple Timestamps are provided, it results in an `AND` condition, enabling the realization of deletions with specified ranges.
  This design allows for versatile deletion operations, facilitating tasks such as removing data within a specific time range.

Output

• the scheme of payload.v1.Object.Locations.

  message Object {
      message Locations { repeated Location locations = 1; }
  
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.Locations

    field	type	label	description
    location	Object.Location	repeated(Array[Object.Location])	The list of Object.Location.

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is removed.
    uuid	string		The ID of the removed vector. It is the same as an Object.ID.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is removed.

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

StreamRemove RPC

StreamRemove RPC is the method to remove multiple vectors using the bidirectional streaming RPC.
Using the bidirectional streaming RPC, the remove request can be communicated in any order between client and server. Each Remove request and response are independent. It’s the recommended method to remove a large number of vectors.

Input

• the scheme of payload.v1.Remove.Request stream

  message Remove {
      message Request {
          Object.ID id = 1;
          Config config = 2;
      }
  
      message Config {
          bool skip_strict_exist_check = 1;
          int64 timestamp = 3;
      }
  }
  
  message Object {
      message ID {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
      }
  }
  

  • Remove.Request

    field	type	label	required	description
    id	Object.ID		*	The ID of vector.
    config	Config		*	The configuration of the insert request.

  • Remove.Config

    field	type	label	required	description
    skip_strict_exist_check	bool			Check whether the same vector is already inserted or not. The ID should be unique if the value is true.
    timestamp	int64			The timestamp of the vector removed. If it is N/A, the current time will be used.

  • Object.ID

    field	type	label	required	description
    id	string		*	The ID of a vector. ID should consist of 1 or more characters.

Output

• the scheme of payload.v1.Object.StreamLocation

  message Object {
      message StreamLocation {
        oneof payload {
            Location location = 1;
            google.rpc.Status status = 2;
        }
      }
  
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.StreamLocation

    field	type	label	description
    location	Object.Location		The information of Object.Location data.
    status	google.rpc.Status		The status of Google RPC

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is removed.
    uuid	string		The ID of the removed vector. It is the same as an Object.ID.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is removed.

  • google.rpc.Status

    field	type	label	description
    code	int32		Status code (code list is next section)
    message	string		Error message
    details	google.protobuf.Any	repeated(Array[any])	The details error message list

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

MultiRemove RPC

MultiRemove is the method to remove multiple vectors in 1 request.

Input

• the scheme of payload.v1.Remove.MultiRequest

  message Remove {
      message MultiRequest {
        repeated Request requests = 1;
      }
  
      message Request {
          Object.ID id = 1;
          Config config = 2;
      }
  
      message Config {
          bool skip_strict_exist_check = 1;
          int64 timestamp = 3;
      }
  }
  
  message Object {
      message ID {
          string id = 1 [ (validate.rules).string.min_len = 1 ];
      }
  }
  

  • Remove.MultiRequest

    field	type	label	required	description
    requests	Remove.Request	repeated(Array[Insert.Request])	*	the request list

  • Remove.Request

    field	type	label	required	description
    id	Object.ID		*	The ID of vector.
    config	Config		*	The configuration of the remove request.

  • Remove.Config

    field	type	label	required	description
    skip_strict_exist_check	bool			Check whether the same vector is already inserted or not. The ID should be unique if the value is true.
    timestamp	int64			The timestamp of the vector removed. If it is N/A, the current time will be used.

  • Object.ID

    field	type	label	required	description
    id	string		*	The ID of a vector. ID should consist of 1 or more characters.

Output

• the scheme of payload.v1.Object.Locations.

  message Object {
      message Locations { repeated Location locations = 1; }
  
      message Location {
        string name = 1;
        string uuid = 2;
        repeated string ips = 3;
      }
  }
  

  • Object.Locations

    field	type	label	description
    location	Object.Location	repeated(Array[Object.Location])	The list of Object.Location.

  • Object.Location

    field	type	label	description
    name	string		The name of vald agent pod where the request vector is removed.
    uuid	string		The ID of the removed vector. It is the same as an Object.ID.
    ips	string	repeated(Array[string])	The IP list of vald-agent pods where the request vector is removed.

Status Code

Please refer to Response Status Code for more details.

Troubleshooting

The request process may not be completed when the response code is NOT 0 (OK).

Here are some common reasons and how to resolve each error.

See also