Custom Norming

To gain meaningful insights from language analysis, it’s essential to interpret scores within the context that makes sense for the language source. While Receptiviti provides norming through the use of our extensive proprietary datasets, custom norming allows you to tailor the norms to your specific dataset or context. For instance, you can create norms based on a specific dataset to generate scores that are directly relevant to that particular data. Alternatively, you can establish norms that apply more broadly to a context, enabling analysis of new datasets within the same contextual framework even if they were not part of the original norming data. This flexibility ensures that normed measures better reflect either localized conditions or broader contextual trends, enhancing the relevance and accuracy of your insights.

Visit our API Reference documentation to learn how to structure your API calls to perform custom norming.

Getting Started

Custom norming of a dataset of language samples is designed for users of the Receptiviti API who wish to establish a median base rate that accurately reflects the average of the dataset’s aggregate.

People talk differently in different contexts. For example, C-suite executives tend to use language with lower frequencies of personal pronouns and first-person singular pronouns such as "I," "me," and "my" than individuals in lower positions and the general population. For reasons like this, it might not be particularly useful or even accurate to compare the way a group of CEOs speak with the patterns of the general population.

Selecting the Right Norming Approach

To determine the best approach for norming your dataset, consider the following factors:

Start by assessing your stored data: If you do not have any stored data, the best option is to use Receptiviti's curated norming datasets, which are designed to provide reliable scores without requiring custom inputs. If you have stored data, proceed to evaluate its similarity to the data you want to analyze.
Evaluate the similarity of your stored data: If your dataset is identical—meaning it was collected under the same conditions, from the same population, and with the same linguistic characteristics as the data you want to analyze—you can confidently proceed with it for norming. However, if your data is only similar, meaning it shares some but not all key characteristics (such as data context, professional jargon, or formality), additional considerations are required to assess whether it provides a reliable baseline. If the stored data is significantly different from your analysis dataset, it is still recommended to use curated norming datasets, as they ensure consistency and validity in interpretation.
Assess the size of your dataset: We recommend working with a robust dataset of text samples for norming, especially when building toward high-confidence outputs. That said, even relatively small datasets may qualify for custom norming in certain contexts — particularly when the use case is well defined or highly domain-specific. Rather than enforce hard thresholds, we encourage you to reach out so we can help assess whether your dataset is a good candidate for custom norming or whether one of our curated norming sets may be more suitable.

info

Larger norming datasets tend to be more representative, making them a more generalizable baseline when analyzing new, similar samples. Smaller norming datasets may be better suited for internal comparisons, where all analyzed samples are also included in the norming dataset. However, when analyzing contexts that are very niche or have inherently small sample and population sizes, a small dataset can still provide meaningful norms, even for unrepresented samples. While custom norming can be valuable, it's not always necessary; if your data closely aligns with an existing norming context, curated norms may be a suitable option. Feel free to contact us at [email protected] for help determining the best approach for your data.

Selecting the Right Norming Context

Choosing the right norming context is necessary for ensuring accurate results. If you have even one text sample, you can still use an existing custom norm to analyze your data with. However, it’s important to note that single samples/small amounts of samples cannot be used to create new norms. Instead, they are best analyzed within a framework of an already created custom norm.

Properly Creating a Custom Norm

Creating a custom norm requires a sufficiently large dataset to ensure the norming process accurately reflects the characteristics of your target context. By supplying a robust dataset during the custom norming process, you can ensure that the resulting norms are both meaningful and effective for subsequent analyses.

Passing Text Samples to the Norming Endpoint

The custom norming endpoint requires you to pass a JSON array of text samples as input. Currently, it does not support CSV uploads directly. To use this endpoint, you need to programmatically prepare your dataset by converting your text samples into a JSON array format. This format ensures the endpoint can process each text sample individually. If your data is stored in a CSV file, you will need to write a script to read the CSV, extract the relevant text, and convert it into the required JSON structure before making the API call. Visit our API Reference documentation to learn more about structuring your data.

Should You Norm Your Dataset?

Follow the guidelines below to get a better understanding of whether or not you should use custom norming for your analyses.

Using the Developer Packages for Custom Norming

The Developer Resources provide a streamlined and efficient way to perform custom norming, making them an excellent choice even if you don’t plan to use the package for other API calls. By leveraging these packages, you can handle custom norming tasks without worrying about managing multiple norming passes or other complexities involved in direct API interactions. This simplicity allows for a smoother, faster setup and execution of norming processes, enabling you to focus on tailoring your data for accurate results.

For R users, go here, and for Python users, go here.

If you're already familiar with using our R or Python packages, here's a basic guide to getting started with the norming tool:

Developer package norming example

To create or interact with a norming context, you'll need to provide some details, such as a name or text to establish the context. For example, you might decide to name your context formal_tone to analyze texts in a formal style later.

Once you've set up a custom norming context, you can use it to analyze future texts by referencing its name in your request. Here's an example:

receptiviti.norming("formal_tone", "/path/to/file.csv", version="v2", text_column="Text")

In this case, formal_tone is the custom context you created earlier, and the system will apply it when analyzing your text.

This tool helps you fine-tune your analysis to specific needs or styles and set specific benchmarks for analyzing future texts.

Use Case Example

Let's say we want to analyze a dataset that contains language from a C-suite executive. Instead of using Receptiviti's default population norms, it would make more sense to measure their language against a group of others who are much more likely to use similar language as the executive in our analysis. In this case, we could use a CEO earnings call dataset as a norming context to ensure that the scores produced by the analysis of the executive reflect a norming dataset that makes more sense contextually.

Once you have your norming context dataset, (i.e., the earnings calls), and you have the language you want to analyze using that earnings calls norming context, you're ready to perform the norming process.

Custom norming is a 3-step process:

Start with a POST call to the endpoint https://api.receptiviti.com/v2/norming/custom. Think of this like the setup phase where you give your norming context a unique name and some information about the word count and sentence structures:
- name (required) gives the custom norming context a name.
- min_word_count (optional) sets the minimum required word count for a text sample to be included in the norming context. Any samples not meeting that threshold are filtered out.
- max_punctuation (optional) sets the maximum amount of punctuation allowed in a sample. If set to 0.5, any sentence whose punctuation makes up 50% of the sentence would be excluded, e.g., This. is. a. test.
Send a PATCH call to the endpoint /v2/norming/{context_name}/one. This requires the entire dataset either in whole or in part (depending on its size). This establishes the norms from our base measures, LIWC and SALLEE.
- The call to this endpoint requires the name of the norming context you created in step 1. If you named your norming context earnings-calls-norms it would be https://api.receptiviti.com/v2/norming/earnings-calls-norms/one.
- Example payload:
```
[
  {
    "text": ""
  },
  {
    "text": "test"
  },
  {
    "text": "This. is. a. test."
  },
  {
    "text": "This is a test."
  },
  {
    "text": "Hello world. Here goes nothing!"
  }
]
```
Send a PATCH call to the endpoint /v2/norming/{context_name}/two. You must pass the exact same dataset here as you did for step 2. This takes the normed base measures from step 1 to enable us to establish the formulas that make up our normed scores, i.e., Big 5, Drives, etc.
- The call to this endpoint would be https://api.receptiviti.com/v2/norming/earnings-calls-norms/two.

At this point, your custom norming context is created. To check which custom norming contexts are present in your account, you can make a GET call to /v2/norming. This will list the contexts you have created as well as any available Receptiviti contexts. A GET call to /v2/norming/custom will list only the custom contexts, including word counts and other metadata.

Now, to analyze the executive using their language against the custom norming context we created above, we would make a POST call to https://api.receptiviti.com/v2/analyze?custom_context=earnings-calls-norms. The response will be the scores for the measures included in our account, normed to the earnings calls dataset we created a norming context for.

A DELETE call to /v2/norming/custom/{context_name} will remove the custom context.

info

See the API Reference section for complete reference documentation for the v2/norming endpoints. Alternatively, you can use our Developer Resources to set custom norming contexts. For R users, go here, and for Python users, go here.

Getting Started​

Selecting the Right Norming Approach​

Selecting the Right Norming Context​

Properly Creating a Custom Norm​

Should You Norm Your Dataset?​

Using the Developer Packages for Custom Norming​

Use Case Example​