Key concepts
Workflow
sequenceDiagram participant User participant Application User->>Application: Load Dataset loop Task Execution User->>Application: Execute Task Application->>Application: Modify State end User->>Application: Save Result
Users can load datasets of any supported file type and apply multiple tasks to manipulate the data. Tasks can operate on selected columns, allowing users to specify input columns and choose output columns for task results. For instance, the “Content/Hash” task takes a user-selected column as input and generates a hash value based on its values, offering a flexible and customizable approach to data transformation and analysis. Read more about tasks
Workflow Concept
Refinator.xyz operates on the principle of a streamlined, task-based workflow reminiscent of the Linux pipe paradigm. This approach allows users to efficiently manipulate and process data by applying individual tasks successively, with the output of each task serving as the input for the subsequent one.
Key Elements
Single Task Execution: Users interact with the application by executing single tasks at a time. Each task is designed to perform a specific operation on the dataset.
State Management
The application dynamically manages the state of the dataset as tasks are executed. The output generated by each task modifies the current state, which becomes the input for the next task in the pipeline.
File-As-Database Concept
Refinator.xyz adopts a unique approach where any file can be treated as a database. Users can load various file formats such as plain text, CSV, TSV, Parquet, and JSON directly into the application. Once loaded, these files serve as the initial dataset, which can be modified and updated iteratively through task execution.
Incremental Updates
Tasks applied to the dataset result in incremental updates to its structure and content. Users can iteratively refine and transform the dataset by chaining together multiple tasks, each building upon the changes made by the preceding ones.