Skip to content

Formatting

ColumnUpdater

Bases: LocalStep

Simple class to create new columns from existing columns in a dataset. Existing columns are not modified.

Parameters:

  • keys_mapping (dict) –

    Dictionary with "from:to" mapping.

Source code in ragfoundry/processing/local_steps/formatting.py 
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
class ColumnUpdater(LocalStep):
    """
    Simple class to create new columns from existing columns in a dataset.
    Existing columns are not modified.

    Args:
        keys_mapping (dict): Dictionary with "from:to" mapping.
    """

    def __init__(self, keys_mapping: dict, **kwargs):
        super().__init__(**kwargs)
        self.keys_mapping = keys_mapping

    def process_item(self, item, index, datasets, **kwargs):
        for from_key, to_key in self.keys_mapping.items():
            item[to_key] = item[from_key]
        return item

FlattenList

Bases: LocalStep

Class to join a list of strings into a single string.

Source code in ragfoundry/processing/local_steps/formatting.py 
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
class FlattenList(LocalStep):
    """
    Class to join a list of strings into a single string.
    """

    def __init__(self, input_key, output_key, string_join=", ", **kwargs):
        """
        Args:
            input_key (str): Key to the list of strings.
            output_key (str): Key to store the joined string.
            string_join (str): String to join the list of strings. Defaults to ", ".
        """
        super().__init__(**kwargs)
        self.input_key = input_key
        self.output_key = output_key
        self.string_join = string_join

    def process_item(self, item, index, datasets, **kwargs):
        item[self.output_key] = self.string_join.join(item[self.input_key])
        return item

__init__(input_key, output_key, string_join=', ', **kwargs)

Parameters:

  • input_key (str) –

    Key to the list of strings.

  • output_key (str) –

    Key to store the joined string.

  • string_join (str, default: ', ' ) –

    String to join the list of strings. Defaults to ", ".

Source code in ragfoundry/processing/local_steps/formatting.py 
28
29
30
31
32
33
34
35
36
37
38
def __init__(self, input_key, output_key, string_join=", ", **kwargs):
    """
    Args:
        input_key (str): Key to the list of strings.
        output_key (str): Key to store the joined string.
        string_join (str): String to join the list of strings. Defaults to ", ".
    """
    super().__init__(**kwargs)
    self.input_key = input_key
    self.output_key = output_key
    self.string_join = string_join