Skip to content

Move source types into their own package and discover them using java.util.ServiceLoader #61

Closed

Description

@tims

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
The ingestion sources are currently mingled in with all the other ingestion code in the transforms package. This makes it hard to distinguish what all the sources are and makes it trickier for people to see how they might extend feast to support new sources.

Describe the solution you'd like
I'd like to create a source package in the ingestion module, and create common factory type class that can create FeatureIO.Read instances and have a function for returning their type string. New source implementations need only implement this interface and use an @AutoService annotation to make it available.

The import job can then load all available source factories from the class path and find the one with the matching type string against the import spec.

This follows the same pattern we use for storage types and allows us a path for including new source types as external dependencies, or moving existing ones into their own module. And then making them available simply by adding their jars on the class path.

Activity

  1. self-assigned this
    on Jan 11, 2019
  2. woop commented on Sep 1, 2019

    @woop
    Member

    This issue isn't relevant any more due to changes in Feast 0.3.0.

  3. added a commit that references this issue on Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions