Documentation Guide¶

This guide covers documentation standards and practices for the Segmentation Robustness Framework.

📝 Documentation Philosophy¶

Documentation Principles¶

Clear and Concise: Write for clarity, not verbosity
User-Focused: Write from the user's perspective
Comprehensive: Cover all public APIs and features
Up-to-Date: Keep documentation synchronized with code
Accessible: Use clear language and proper formatting

Documentation Types¶

API Documentation: Function and class documentation
User Guides: How-to guides and tutorials
Conceptual Documentation: Architecture and design explanations
Examples: Code examples and use cases
Reference Documentation: Complete API reference

📚 Documentation Structure¶

docs/
├── index.md                    # Main documentation page
├── installation.md             # Installation guide
├── quick_start.md             # Quick start tutorial
├── user_guide.md              # Comprehensive user guide
├── core_concepts.md           # Framework concepts
├── api_reference/             # API documentation
│   ├── index.md              # API overview
│   ├── pipeline.md           # Pipeline API
│   ├── attacks.md            # Attacks API
│   ├── metrics.md            # Metrics API
│   └── adapters.md           # Adapters API
├── examples/                  # Code examples
│   ├── basic_examples.md     # Basic usage examples
│   └── advanced_examples.md  # Advanced examples
├── contributing/              # Contributing documentation
│   ├── index.md              # Contributing guide
│   ├── development_setup.md  # Development setup
│   ├── testing_guide.md      # Testing guidelines
│   └── documentation_guide.md # This file
└── img/                      # Documentation images
    ├── architecture.png      # Framework architecture
    └── workflow.png          # Usage workflow

✍️ Writing Documentation¶

Documentation Standards¶

Markdown Formatting¶

# Main Heading (H1)

## Section Heading (H2)

### Subsection Heading (H3)

#### Sub-subsection Heading (H4)

**Bold text** for emphasis

*Italic text* for secondary emphasis

`code` for inline code

```python
# Code blocks with syntax highlighting
def example_function():
    return "Hello, World!"
```

> Blockquotes for important notes or warnings

Code Examples¶

# Good example - complete and runnable
from segmentation_robustness_framework.pipeline import SegmentationRobustnessPipeline
from segmentation_robustness_framework.attacks import FGSM

# Load model
model = load_model("deeplabv3_resnet50")

# Create attack
attack = FGSM(model, eps=0.02)

# Create pipeline
pipeline = SegmentationRobustnessPipeline(
    model=model,
    dataset=dataset,
    attacks=[attack],
    metrics=metrics,
    batch_size=4
)

# Run evaluation
results = pipeline.run()

API Documentation¶

def evaluate_model(
    model: SegmentationModelProtocol,
    dataset: torch.utils.data.Dataset,
    metrics: list[Callable]
) -> dict[str, float]:
    """Evaluate a segmentation model.

    This function evaluates a segmentation model on a dataset using
    specified metrics. The evaluation includes both clean and adversarial
    performance.

    Args:
        model: The segmentation model to evaluate. Must implement the
            SegmentationModelProtocol interface.
        dataset: Dataset for evaluation. Should return (image, mask) pairs.
        metrics: List of metric functions. Each function should accept
            (targets, predictions) and return a float.

    Returns:
        Dictionary containing evaluation results with keys:
            - 'clean': Clean performance metrics
            - 'attack_<name>': Attack-specific metrics

    Raises:
        ValueError: If model or dataset is invalid.
        RuntimeError: If evaluation fails.

    Example:
        ```python
        model = load_model("deeplabv3_resnet50")
        dataset = VOCSegmentation(split="val")
        metrics = [mean_iou, pixel_accuracy]

        results = evaluate_model(model, dataset, metrics)
        print(f"Clean IoU: {results['clean']['mean_iou']:.3f}")
        ```
    """
    # Implementation here
    pass

Documentation Style Guide¶

Tone and Voice¶

Use active voice: "The function returns..." not "The function is returned by..."
Be direct and clear: Avoid unnecessary words
Use present tense: "The function processes..." not "The function will process..."
Be consistent: Use consistent terminology throughout

Language Guidelines¶

Use US English: color, center, analyze
Avoid jargon: Explain technical terms
Be inclusive: Use inclusive language
Be precise: Use exact terms and measurements

Formatting Guidelines¶

Use proper headings: H1 for page title, H2 for sections, etc.
Use lists appropriately: Bullet points for unordered lists, numbers for steps
Use code blocks: For all code examples
Use emphasis sparingly: Bold for important terms, italic for emphasis

📖 API Documentation¶

Function Documentation¶

def process_images(
    images: torch.Tensor,
    target_size: tuple[int, int],
    normalize: bool = True
) -> torch.Tensor:
    """Process images to target size.

    Resize and optionally normalize images to the specified target size.
    The function supports both CPU and GPU tensors.

    Args:
        images: Input images [B, C, H, W]. Can be on CPU or GPU.
        target_size: Target size as (height, width) in pixels.
        normalize: Whether to normalize images to [0, 1] range.
            Defaults to True.

    Returns:
        Processed images [B, C, H, W] with same device as input.

    Raises:
        ValueError: If target_size contains non-positive values.
        RuntimeError: If image processing fails.

    Example:
        ```python
        images = torch.randn(4, 3, 224, 224)
        processed = process_images(images, (512, 512))
        assert processed.shape == (4, 3, 512, 512)
        ```

    Note:
        This function modifies images in-place for efficiency.
    """
    # Implementation
    pass

Class Documentation¶

class SegmentationRobustnessPipeline:
    """Pipeline for evaluating segmentation models under adversarial attacks.

    This pipeline provides a unified interface for evaluating segmentation
    models on clean and adversarial images. It supports multiple attacks,
    metrics, and output formats.

    The pipeline automatically handles:
    - Model evaluation on clean images
    - Attack generation and evaluation
    - Metric computation and aggregation
    - Result saving and visualization

    Attributes:
        model: The segmentation model to evaluate.
        dataset: Dataset for evaluation.
        attacks: List of attack instances.
        metrics: List of metric functions.
        batch_size: Batch size for evaluation.
        device: Device to use for computation.
        output_dir: Directory to save results.

    Example:
        ```python
        pipeline = SegmentationRobustnessPipeline(
            model=model,
            dataset=dataset,
            attacks=[FGSM(model, eps=0.02)],
            metrics=[mean_iou, pixel_accuracy],
            batch_size=4,
            device="cuda"
        )

        results = pipeline.run()
        pipeline.print_summary()
        ```
    """

    def __init__(
        self,
        model: SegmentationModelProtocol,
        dataset: torch.utils.data.Dataset,
        attacks: list,
        metrics: list[Callable],
        batch_size: int = 8,
        device: str = "cpu",
        output_dir: Optional[str] = None
    ):
        """Initialize the segmentation robustness pipeline.

        Args:
            model: Segmentation model (adapter-wrapped).
            dataset: Dataset object for evaluation.
            attacks: List of attack instances.
            metrics: List of metric functions or classes.
            batch_size: Batch size for evaluation. Defaults to 8.
            device: Device to use for computation. Defaults to "cpu".
            output_dir: Directory to save results. If None, uses "./runs".
        """
        # Implementation
        pass

Module Documentation¶

"""Segmentation Robustness Framework - Attacks Module.

This module provides implementations of adversarial attacks for semantic
segmentation models. All attacks follow a common interface and can be
used interchangeably in the evaluation pipeline.

Available Attacks:
    - FGSM: Fast Gradient Sign Method
    - PGD: Projected Gradient Descent
    - RFGSM: R-FGSM with momentum
    - TPGD: Two-Phase Gradient Descent

Example:
    ```python
    from segmentation_robustness_framework.attacks import FGSM, PGD

    # Create attacks
    fgsm_attack = FGSM(model, eps=0.02)
    pgd_attack = PGD(model, eps=0.02, alpha=0.01, iters=10)

    # Use in pipeline
    pipeline = SegmentationRobustnessPipeline(
        model=model,
        dataset=dataset,
        attacks=[fgsm_attack, pgd_attack],
        metrics=metrics
    )
    ```
"""

# Module implementation

📋 User Guide Documentation¶

Structure¶

Introduction: What the feature does and why it's useful
Prerequisites: What users need to know or have
Step-by-step instructions: Clear, numbered steps
Examples: Real-world usage examples
Troubleshooting: Common issues and solutions

Example User Guide Section¶

## Adding Custom Attacks

This guide shows you how to create and use custom adversarial attacks
with the Segmentation Robustness Framework.

### Prerequisites

- Basic knowledge of PyTorch
- Understanding of adversarial attacks
- Familiarity with the framework's attack interface

### Step 1: Create Your Attack Class

First, create a new attack class that inherits from `AdversarialAttack`:

```python
from segmentation_robustness_framework.attacks import AdversarialAttack
from segmentation_robustness_framework.attacks.registry import register_attack

@register_attack("custom_attack")
class CustomAttack(AdversarialAttack):
    """Custom adversarial attack implementation."""

    def __init__(self, model: nn.Module, eps: float = 0.02):
        super().__init__(model)
        self.eps = eps

    def apply(self, images: torch.Tensor, labels: torch.Tensor) -> torch.Tensor:
        """Apply custom attack to images."""
        # Your attack implementation here
        return adversarial_images
```

### Step 2: Implement the Attack Logic

The `apply` method should:
- Take input images and labels
- Generate adversarial perturbations
- Return adversarial images

### Step 3: Use Your Attack

Register and use your attack in the pipeline:

```python
# Create attack instance
custom_attack = CustomAttack(model, eps=0.03)

# Use in pipeline
pipeline = SegmentationRobustnessPipeline(
    model=model,
    dataset=dataset,
    attacks=[custom_attack],
    metrics=metrics
)

results = pipeline.run()
```

### Examples

#### Simple Random Attack

```python
@register_attack("random_attack")
class RandomAttack(AdversarialAttack):
    """Simple random perturbation attack."""

    def __init__(self, model: nn.Module, eps: float = 0.02):
        super().__init__(model)
        self.eps = eps

    def apply(self, images: torch.Tensor, labels: torch.Tensor) -> torch.Tensor:
        """Apply random perturbation."""
        perturbation = torch.randn_like(images) * self.eps
        adv_images = torch.clamp(images + perturbation, 0, 1)
        return adv_images
```

### Troubleshooting

**Issue**: Attack not registered
- **Solution**: Make sure to use the `@register_attack` decorator

**Issue**: Attack produces invalid images
- **Solution**: Ensure images are in [0, 1] range using `torch.clamp`

**Issue**: Attack is too slow
- **Solution**: Use vectorized operations and avoid loops

🖼️ Visual Documentation¶

Screenshots and Diagrams¶

Use high-quality images: PNG format, appropriate resolution
Include alt text: For accessibility
Keep images up-to-date: Update when UI changes
Use consistent styling: Same color scheme and fonts

🔄 Documentation Maintenance¶

Keeping Documentation Current¶

Update with code changes: Modify docs when code changes
Review regularly: Schedule regular documentation reviews
Test examples: Ensure all code examples work
Check links: Verify all internal and external links
Update version numbers: Keep version info current

Documentation Review Process¶

Self-review: Review your own documentation
Peer review: Have others review your documentation
User testing: Test with actual users
Continuous improvement: Gather feedback and improve

Documentation Tools¶

# Build documentation locally
mkdocs build

# Serve documentation locally
mkdocs serve

# Check for broken links
linkchecker docs/

# Validate markdown
markdownlint docs/

API Reference Generation¶

The API reference documentation is automatically generated from docstrings using MkDocs and the mkdocstrings plugin. This ensures that the API documentation stays synchronized with the code.

Setup for API Documentation¶

# Install mkdocstrings plugin
pip install mkdocstrings[python]

# Install additional plugins for better documentation
pip install mkdocs-material
pip install mkdocs-git-revision-date-localized-plugin

Configuration¶

The API documentation is configured in mkdocs.yml:

plugins:
  - search
  - mkdocstrings:
      default_handler: python
      handlers:
        python:
          paths: [segmentation_robustness_framework]
          options:
            show_source: true
            show_root_heading: true
            show_category_heading: true
            show_signature_annotations: true
            show_bases: true
            heading_level: 1
            members_order: source
            docstring_style: google
            preload_modules: true

Generating API Reference¶

# Generate API documentation from docstrings
mkdocs build

# The API reference will be automatically generated in:
# - docs/api_reference/pipeline.md
# - docs/api_reference/attacks.md
# - docs/api_reference/metrics.md
# - docs/api_reference/adapters.md

Docstring Requirements for API Generation¶

To ensure proper API documentation generation, follow these docstring standards:

Use Google-style docstrings (required by mkdocstrings)
Include type hints in function signatures
Document all parameters with types and descriptions
Document return values with types
Document exceptions that may be raised
Include usage examples in docstrings

Example of a well-documented function for API generation:

def evaluate_model(
    model: SegmentationModelProtocol,
    dataset: torch.utils.data.Dataset,
    metrics: list[Callable[[torch.Tensor, torch.Tensor], float]]
) -> dict[str, float]:
    """Evaluate a segmentation model on a dataset.

    This function evaluates a segmentation model on a dataset using
    specified metrics. The evaluation includes both clean and adversarial
    performance.

    Args:
        model: The segmentation model to evaluate. Must implement the
            SegmentationModelProtocol interface.
        dataset: Dataset for evaluation. Should return (image, mask) pairs.
        metrics: List of metric functions. Each function should accept
            (targets, predictions) and return a float.

    Returns:
        Dictionary containing evaluation results with keys:
            - 'clean': Clean performance metrics
            - 'attack_<name>': Attack-specific metrics

    Raises:
        ValueError: If model or dataset is invalid.
        RuntimeError: If evaluation fails.

    Example:
        ```python
        model = load_model("deeplabv3_resnet50")
        dataset = VOCSegmentation(split="val")
        metrics = [mean_iou, pixel_accuracy]

        results = evaluate_model(model, dataset, metrics)
        print(f"Clean IoU: {results['clean']['mean_iou']:.3f}")
        ```
    """
    # Implementation
    pass

API Documentation Structure¶

The generated API documentation follows this structure:

docs/api_reference/
├── index.md              # API overview and navigation
├── pipeline.md           # Pipeline classes and functions
├── attacks.md            # Attack implementations
├── metrics.md            # Metric functions and classes
├── adapters.md           # Model adapters
├── datasets.md           # Dataset classes
└── loaders.md            # Model and dataset loaders

Customizing API Documentation¶

You can customize the API documentation generation by:

Adding custom handlers in mkdocs.yml
Filtering modules to include/exclude specific components
Customizing templates for different documentation styles
Adding cross-references between related functions

Example customization:

plugins:
  - mkdocstrings:
      default_handler: python
      handlers:
        python:
          options:
            show_source: true
            show_root_heading: true
            show_category_heading: true
            show_signature_annotations: true
            show_bases: true
            heading_level: 1
            members_order: source
            docstring_style: google
            preload_modules: true
            filters: ["!^_"]  # Exclude private members
            members: ["public", "special"]  # Include public and special methods

Maintaining API Documentation¶

To keep API documentation up-to-date:

Update docstrings when changing function signatures
Add new functions with proper docstrings
Run documentation build to regenerate API reference
Review generated documentation for accuracy
Test documentation locally before committing

# Regenerate API documentation
mkdocs build

# Check for documentation errors
mkdocs build --strict

# Serve documentation locally for review
mkdocs serve

🚀 Best Practices¶

Writing Guidelines¶

Start with the user: Write from the user's perspective
Be specific: Use concrete examples and numbers
Use active voice: "The function returns..." not "The function is returned by..."
Be consistent: Use consistent terminology and formatting
Test your examples: Ensure all code examples work

Organization Guidelines¶

Logical structure: Organize content logically
Progressive disclosure: Start simple, add complexity
Cross-references: Link related content
Searchable: Use clear headings and keywords
Scannable: Use lists, tables, and formatting

Maintenance Guidelines¶

Version control: Track documentation changes
Regular reviews: Schedule documentation reviews
User feedback: Gather and incorporate feedback
Automation: Use tools to check documentation
Continuous improvement: Always look for ways to improve

🎯 Next Steps¶

After reading this documentation guide:

Review existing documentation to understand current standards
Practice writing documentation using the templates provided
Contribute documentation for new features or improvements
Help maintain documentation by reporting issues or suggesting improvements
Share knowledge with other contributors

Happy documenting! 📚