Skip to content

Filters

Module containing filters.

msmarco_positive_filter(x)

Identify the positive passages in MSMARCO dataset.

Source code in ragfit/processing/global_steps/filters.py 
4
5
6
7
8
def msmarco_positive_filter(x):
    """
    Identify the positive passages in MSMARCO dataset.
    """
    return 1 in x["passages"]["is_selected"]