> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/gradio-app/gradio/llms.txt
> Use this file to discover all available pages before exploring further.

# Flagging

> Learn how to let users flag interesting or unexpected outputs in your Gradio interfaces

The "Flag" button allows users to mark inputs with interesting outputs, such as erroneous or unexpected model behavior, for you to review later.

## How flagging works

When a user flags data in your Interface:

1. A CSV file logs the flagged inputs and outputs
2. If the interface involves file data (images, audio, etc.), folders are created to store those files
3. All flagged data is saved in the directory specified by `flagging_dir`

## Basic flagging setup

The flag button appears by default in your `Interface`. You can control its behavior:

```python theme={null}
import gradio as gr

def calculator(num1, operation, num2):
    if operation == "add":
        return num1 + num2
    elif operation == "subtract":
        return num1 - num2
    elif operation == "multiply":
        return num1 * num2
    elif operation == "divide":
        if num2 == 0:
            raise gr.Error("Cannot divide by zero!")
        return num1 / num2

demo = gr.Interface(
    calculator,
    inputs=["number", gr.Radio(["add", "subtract", "multiply", "divide"]), "number"],
    outputs="number",
    flagging_mode="manual"  # Users click flag button to flag
)

demo.launch()
```

## Flagging modes

You can control when and how flagging occurs with the `flagging_mode` parameter:

<AccordionGroup>
  <Accordion title="manual (default)">
    Users see a flag button and must click it to flag data:

    ```python theme={null}
    gr.Interface(
        fn=model,
        inputs=[...],
        outputs=[...],
        flagging_mode="manual"
    )
    ```
  </Accordion>

  <Accordion title="auto">
    Every submission is automatically flagged along with its output:

    ```python theme={null}
    gr.Interface(
        fn=model,
        inputs=[...],
        outputs=[...],
        flagging_mode="auto"
    )
    ```

    Users won't see a flag button - all interactions are logged automatically.
  </Accordion>

  <Accordion title="never">
    Flagging is completely disabled:

    ```python theme={null}
    gr.Interface(
        fn=model,
        inputs=[...],
        outputs=[...],
        flagging_mode="never"
    )
    ```
  </Accordion>
</AccordionGroup>

<Note>
  You can also set the `GRADIO_FLAGGING_MODE` environment variable to control flagging mode globally.
</Note>

## Flagged data structure

For a simple calculator interface, flagged data is stored like this:

```
+-- calculator.py
+-- flagged/
|   +-- logs.csv
```

**flagged/logs.csv:**

```csv theme={null}
num1,operation,num2,Output
5,add,7,12
6,subtract,1.5,4.5
```

For interfaces with file inputs/outputs (like images), the structure includes data folders:

```
+-- sepia.py
+-- flagged/
|   +-- logs.csv
|   +-- im/
|   |   +-- 0.png
|   |   +-- 1.png
|   +-- Output/
|   |   +-- 0.png
|   |   +-- 1.png
```

**flagged/logs.csv:**

```csv theme={null}
im,Output
im/0.png,Output/0.png
im/1.png,Output/1.png
```

## Custom flagging directory

Specify where flagged data is stored:

```python theme={null}
gr.Interface(
    fn=model,
    inputs=[...],
    outputs=[...],
    flagging_dir="./my_flagged_data"
)
```

The directory is created automatically if it doesn't exist.

## Flagging options

Allow users to provide a reason when flagging by passing a list of strings to `flagging_options`:

```python theme={null}
import gradio as gr

def image_classifier(img):
    return {"cat": 0.7, "dog": 0.3}

demo = gr.Interface(
    image_classifier,
    inputs="image",
    outputs="label",
    flagging_mode="manual",
    flagging_options=["Incorrect", "Offensive", "Other"]
)

demo.launch()
```

Users select one option when flagging, and it's saved as an additional column in the CSV:

```csv theme={null}
im,Output,flag_option
im/0.png,"{'cat': 0.7, 'dog': 0.3}",Incorrect
im/1.png,"{'cat': 0.2, 'dog': 0.8}",Other
```

## Custom flag labels and values

You can customize both the button labels and the values stored in the CSV:

```python theme={null}
gr.Interface(
    fn=model,
    inputs=[...],
    outputs=[...],
    flagging_options=[
        ("Mark as incorrect", "incorrect"),
        ("Mark as offensive", "offensive"),
        ("Mark as problematic", "problematic")
    ]
)
```

The first element of each tuple is the button label shown to users, and the second is the value stored in the CSV.

## Custom flagging callbacks

By default, Gradio uses `CSVLogger` to save flagged data. You can create custom flagging behavior by providing a `FlaggingCallback` subclass:

```python theme={null}
import gradio as gr
from gradio.flagging import FlaggingCallback

class SimpleFlaggingCallback(FlaggingCallback):
    def setup(self, components, flagging_dir):
        self.components = components
        self.flagging_dir = flagging_dir
        # Setup logic (create directories, initialize database, etc.)
    
    def flag(self, flag_data, flag_option=None, username=None):
        # Custom flagging logic (save to database, send notification, etc.)
        print(f"Flagged: {flag_data}")
        if flag_option:
            print(f"Reason: {flag_option}")
        return len(flag_data)  # Return count of flagged samples

demo = gr.Interface(
    fn=model,
    inputs="image",
    outputs="label",
    flagging_callback=SimpleFlaggingCallback()
)

demo.launch()
```

<Accordion title="FlaggingCallback interface">
  Your custom callback must implement two methods:

  **setup(components, flagging\_dir)**

  * Called once at the beginning of `Interface.launch()`
  * `components`: List of input/output components
  * `flagging_dir`: Directory path for storing flagged data

  **flag(flag\_data, flag\_option=None, username=None)**

  * Called every time the flag button is pressed
  * `flag_data`: List of data to be flagged
  * `flag_option`: Selected option if `flagging_options` provided
  * `username`: Username if user is logged in
  * Returns: Total number of flagged samples (int)
</Accordion>

## SimpleCSVLogger example

Gradio provides `SimpleCSVLogger` as a basic example:

```python theme={null}
import gradio as gr

def image_classifier(inp):
    return {'cat': 0.3, 'dog': 0.7}

demo = gr.Interface(
    fn=image_classifier,
    inputs="image",
    outputs="label",
    flagging_callback=gr.flagging.SimpleCSVLogger()
)

demo.launch()
```

This is a simplified implementation provided for illustrative purposes. The default `CSVLogger` has more features.

## Loading flagged data as examples

You can use flagged data as examples by pointing to the flagged directory:

```python theme={null}
gr.Interface(
    fn=model,
    inputs=[...],
    outputs=[...],
    examples="./flagged"  # Load from flagged directory
)
```

This is helpful for reviewing and demonstrating edge cases.

## Next steps

<CardGroup cols={2}>
  <Card title="Interface class" icon="cube" href="/guides/interface-class">
    Learn more about the Interface class
  </Card>

  <Card title="Examples" icon="code" href="/guides/examples">
    Working with examples in interfaces
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.