Skip to content

Simplify DNN dense corpus and interaction with python backend #88

Description

@johann-petrak
  • see also IMPORTANT! Refactor and simplify gate-lf-python-data#15
  • Keep the option to have many features but make it easy to have just the simple one-feature approach.
  • Store dense corpus instances as maps in each line with standard keys for label and possibly features
  • unify with representation for unlabeled data (e.g. embedding creation or topic models) and other kinds of supervised/unsupervised tasks, e.g. seq2seq or semantic similarity
  • !!!! change representation of sequences: instead of having a sequence of element with multiple features, have a sequence for each feature. Makes it MUCH easier to create batches later.
  • Make it easy to swith between our output and the torchnlp library in the python backend

This should become a project possibly with several subissues.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions