> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getmembrane.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Collections

Data collections allow working with data in an external application using a consistent API. They let you do the following:

* Get schema of data record fields, including custom fields configured for a particular connection.
* Perform CRUD operations on data records using a consistent API.
* Understand which operations are possible and not possible for a given collection.
* Subscribe to data change events even if webhooks are not supported by the underlying API.

To learn how to work with data collections, see [Data Collections API Reference](/reference/workspace-elements/data-collections)

<br />

<Callout icon="ℹ️" theme="info">
  Data Collections are connector-level interfaces. To change them, you need to modify the
  [Connector](/reference/workspace-elements/connectors) that implements a given data collection.
</Callout>

## Spec

The data collection specification defines the structure and capabilities of a data collection.

It is located in the root of the data collection directory (`data/<collection-key>/spec.yml`).

Here's an example:

```yaml theme={null}
contacts:
  name: Contacts
  fieldsSchema:
    type: object
    properties:
      id:
        type: string
        readOnly: true
      email:
        type: string
      firstName:
        type: string
      lastName:
        type: string
      fullName:
        type: string
        writeOnly: true
  methods:
    list:
      filterFields:
        - email
    create:
      requiredFields:
        - email
    update:
      excludedFields:
        - id
    delete: {}
    findById: {}
    match:
      fields:
        - email
    search: {}
  events:
    created:
      implementationType: webhook
    updated:
      implementationType: pull-latest-records
    deleted:
      implementationType: custom-pull
```

## Properties

| Property           | Type   | Description                                                                                      |
| ------------------ | ------ | ------------------------------------------------------------------------------------------------ |
| `name`             | string | Display name for the collection                                                                  |
| `parametersSchema` | object | [Data Schema](/docs/references/data-schemas) for collection-level parameters                     |
| `fieldsSchema`     | object | [Data Schema](/docs/references/data-schemas) defining the structure of records in the collection |
| `methods`          | object | Method implementations. See below for details.                                                   |
| `events`           | object | Event implementations. See below for details.                                                    |

### Parameters

Data collection parameters are used to configure the data collection implementation.
They are passed to execution context of all data collection functions as `parameters` argument.

You should only add parameters to the data collection when you cannot use function-level parameters.

If collection-level and function-level parameters overlap, function-level parameters take precedence.

## Fields Schema

This [Data Schema](/docs/references/data-schemas) defines fields of the records in this data collection.

It is recommended that all the fields that could participate in any operation of this collection be present in the fields schema. If some fields are only available in read or write operations, you can mark them as `readOnly` or `writeOnly`.

Furthermore, you can configure which fields are available for which operations (i.e. `create` or `update`) in the corresponding operation configuration.

**Custom Fields**

If a data collection supports custom fields that can be configured on a per-connection basis, you can implement a `customFields` function that returns the custom fields schema.\\

The custom fields schema will be merged with `fieldsSchema` when returning the collection's specification.

## Methods

You can configure the following methods for the data collection:

| Method                                                                                                          | Description                                                                                |
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| [`list`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-list)           | Read all the records from the collection (with optional filters).                          |
| [`findById`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-find-by-id) | Get one record from the collection using its id.                                           |
| [`match`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-match)         | Find a single matching record in a data collection using provided fields (e.g., by email). |
| [`search`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-search)       | Find a list of records using a string query. Supports type-ahead search whenever possible. |
| [`create`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-create)       | Create one record in a data collection.                                                    |
| [`update`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-update)       | Update a record in a data collection using its id.                                         |
| [`delete`](/reference/workspace-elements/data-collections/data-collection-methods/data-collection-delete)       | Delete a record with a given id.                                                           |

## Data Collection Functions

These are non-method functions that affect the data collection functionality.

| Function                                                                                                | Description                                                                 |
| ------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| [`recordFromFields`](/reference/workspace-elements/data-collections/data-collection-record-from-fields) | Build Data Records from their fields - every data collection must have one  |
| [`spec`](/reference/workspace-elements/data-collections/data-collection-dynamic-spec)                   | Dynamically generate the collection specification based on API capabilities |
| [`customFields`](/reference/workspace-elements/data-collections/custom-fields)                          | Dynamically generate custom fields schema based on API configuration        |
| [`fieldsFromApi`](/reference/workspace-elements/data-collections/data-collection-fields-from-api)       | Transform API response fields to match your fieldsSchema format             |
| [`fieldsToApi`](/reference/workspace-elements/data-collections/data-collection-fields-to-api)           | Transform fieldsSchema format to API format before sending requests         |

When you want to transform fields that are coming from the API before returning them from the connector, or transform fields that come from the client before sending them to the API, you can use `fieldsFromApi` and `fieldsToApi` functions respectively.

These functions will be automatically applied to fields for all method and event implementations.

`fieldsToApi` will be automatically applied after method is executed and before sending the request to the API.
`fieldsFromApi` will be automatically applied after the method is executed and before the result is returned to the client.

This could be useful if:

* Fields in the API are inconsistent between endpoints.
* Field structures in the API are too complex and you want to simplify them.

## Inheritance (Mixins)

When multiple data collections share the implementation of some of their functions, you can use mixins.

Example:

```yaml theme={null}
# spec.yml
name: Users
extends:
  - key: _webhooks
    parameters:
      objectKey: user
```

If a collection uses a mixin, all the functions (and methods) from the mixin will be used as the collection's functions unless overridden.

If a mixin collection has parameters, each collection that uses the mixin can provide different parameter values.