tf.contrib.lookup.TextFileIdTableInitializer

class tf.contrib.lookup.TextFileIdTableInitializer

Table initializer for string to int64 IDs tables from a text file.

Properties

key_dtype

The expected table key dtype.

value_dtype

The expected table value dtype.

Methods

__init__(filename, key_column_index=TextFileIndex.WHOLE_LINE, value_column_index=TextFileIndex.LINE_NUMBER, vocab_size=None, delimiter='\t', name='text_file_id_table_init')

Constructs an initializer for an string-to-id table from a text file.

It populates a table that its key and value types are string and int64, respectively. It generates one key-value pair per line. The content of the key and value are specified by the key_index and value_index.

• TextFileIndex.LINE_NUMBER means use the line number starting from zero, expects data type int64.
• TextFileIndex.WHOLE_LINE means use the whole line content, expects data type string.
• A value >=0 means use the index (starting at zero) of the split line based on delimiter.

Args:

• filename: The filename of the text file to be used for initialization. The path must be accessible from wherever the graph is initialized (eg. trainer or eval workers). The filename may be a scalar Tensor.
• key_column_index: The column index from the text file to get the key values from. The default is to use the line number, starting from zero.
• value_column_index: The column index from the text file ro get the value values from. The default is 0 that represents the whole line content.
• vocab_size: The number of elements in the file, if known.
• delimiter: The delimiter to separate fields in a line.
• name: Optional name for the op.

Raises:

• TypeError: when the filename is empty, or when the table key and value data types do not match the expected data types.

initialize(table)

Initializes the table from a text file.

Args:

• table: The table to be initialized.

Returns:

The operation that initializes the table.

Raises:

• TypeError: when the keys and values data types do not match the table key and value data types.

Defined in tensorflow/contrib/lookup/lookup_ops.py.