Tuning Search Performance

ANN is fast, but sometimes it can be improved more by tuning parameters. This page shows the essence for improve ANN search by the Vald cluster.

Tuning Guideline

When the search results do NOT satisfy the expected result, it can be improved by tuning parameters.

First of all, we recommend tuning by following the steps below without doing it blindly.

flowchart TD
    A[Perform Linear Search API]
    B{Is satisfies?}
    C[Tuning Parameters to improve precision]
    D[Tuning Embedding Models]
    E[Perform Search API]
    F[Tuning Parameters to improve latency]
    B-- Yes -->C
    B-- No -->D
    C--> E
    E--> C
    E--> F
    F--> E

The best practice is:

  1. Measure Linear Search performance and use it as a baseline for Search API
  2. Repeat tuning to improve precision and measure Search API until the conditions are met
  3. Repeat tuning to improve latency and measure Search API until the conditions are met
When the results are not good by Linear Search API, it may need to rethink the embedding model for vectorization.

Tuning parameters

There are two viewpoints, client-side and cluster-side, for improving search performance.

There is a trade-off between search speed and accuracy, so tuning accuracy at first is recommended.

Client side

On the client side, parameters of Search.Config will affect the search result.

descriptionhow does it affect?memo
radiusthe search radius for NGT
ref: search_radius
Define the search range when NGT searches the nearest neighborsrecommended value: -1
epsilonthe search coefficient for NGT
ref: search_range_coefficient
Expansion factor of the NGT search range.
Search operation time increases when the epsilon is big.
recommended value range: 0.01 ~ 0.3
default value: 0.1
timeout(ns)max time duration until receiving search results.An error will be returned if the set num search results cannot be obtained within the set time.
By setting min_num, the search results will be returned if more than min_num can be searched within the time.
default value: 3,000,000,000ns


On the cluster side, these parameters can be set by values.yaml, affect the search result.

descriptionhow does it affect?Memo
agent.ngt.creation_edge_sizeNumber of nodes connected to one node
ref: no_of_edges
It helps reduce unreachable edges.
The larger it is, the denser the graph structure will be, but the memory usage, search speed, and index construction time will increase accordingly.
default value: 20
agent.ngt.search_edge_sizeNumber of nodes to search from the origin node when searching
ref: no_of_edges_at_search_at
The number of nodes to search will increase.
Accuracy will be higher, but speed will be lower.
Adjust if adjusting the radius and epsilon does not improve the situation.
default value: 10

See also