Create Dataset

Data uses the Dataloader to load and parse the raw offline data and obtain the transition data containing (observation, action, reward, done), and then stores the obtained transition into the ReplayBuffer, and finally obtains the Dataset required for offline RL training. The base class of Data, BaseData, contains the following parts:

  • __init__(): Contains some initialization information, such as data file path, dataloader attributes, Buffer size, etc.;

  • _build_data_loader(): Construct DataLoader;

  • _build_data(): Add the transitions obtained from Dataloader to Buffer to obtain Dataset;

  • data(): Call this interface to return the final Replay Buffer, which is the constructed dataset.

Data

Data inherits from BaseData and can use different Dataloaders to load offline data from different sources and application types to construct the dataset required for training. Here are the introductions of each part:

  • __init__(): Based on the parameter information in the configuration file, determine the type of Dataloader;

  • _app_data_loader(): Construct a Dataloader for real-world scenario data;

  • _d4rl_data_loader(): Construct a Dataloader for the D4RL dataset;

  • _data_loader_list(): Put the above-mentioned Dataloaders into a dictionary;

In addition to the two Dataloaders mentioned above, you can also add custom Dataloaders for other benchmark datasets, just add a new dataloader construction method in Data, and add the new method to the dictionary in _data_loader_list().

In addition, you can also customize other Data according to your needs to achieve customized requirements, such as DataNoise inherits from Data and re-implements the _build_data() method to construct a Dataset that adds noise to the action. Another example is DataMix, which inherits from BaseData and implements a dataset that mixes multiple data sources.

Customize Dataloader

By inheriting from BaseDataLoader, different types of Dataloaders can be implemented to import offline data from different sources. The BaseDataLoader mainly contains the following parts:

  • _load_data(): Load data from the raw data file and return transitions elements;

  • get_transitions(): Process data and generate transitions;

  • state_shift_scale(): Get the shift and scale of the state normalization.

When customizing the Dataloader for benchmark datasets, you can inherit from the base class BaseBMLoader and implement _load_data() to construct Dataloaders for different benchmark datasets. D4rlDataLoader is a Dataloader constructed for the D4RL dataset, which can be used as a reference.

For offline data from real-world scenarios, they are usually saved as .csv files. AppDataLoader is specially designed to load offline datasets from real-world scenarios.