LL-012: Trusted-Source RSS/Atom Ingestion Specification

Objective Define the structural requirements for ingesting high-trust feeds via RSS or Atom protocols, ensuring provenance metadata is captured and displayed distinctly from generated content.

Scope & Standards

• Protocol Adherence: Ingestion endpoints must comply with RFC 4379 (Atom 1.0) or RFC 3298 (RSS 0.9+) for parsing stability.
• Provenance Compliance: Follows ISO/IEC 27001 data governance principles regarding source integrity and immutability.

Allowlist Configuration

• Storage Location: /config/ingestion/trusted_feeds.yaml.
• Validation Logic: The system validates every incoming feed URI against this allowlist prior to ingestion. If a URI is not present, the feed entry remains unprocessed in the database.
• Required Fields: The ingest pipeline must extract source, title, pub_date, and external_url from every item.

Feed Display Schema Every item ingested via this channel must display a dedicated badge indicating "Trusted Source" in addition to the standard metadata fields.

• Header Fields:
• Source: Name of the publisher (e.g., IEEE, NIST).
• Type: RSS or Atom.
• Date: Original publication timestamp converted to UTC.
• Title: Feed item title.
• Link: Canonical external URL to the content.

Provenance Storage Requirements

• Storage Class: Items must be stored in a read-only provenance archive to prevent accidental modification of source metadata.
• Retrieval: All items ingested post-deployment are accessible via an internal audit interface for compliance verification.

LL-013: Open Net Discovery Feed Specification

Objective Establish visual and editorial separation between open-web automated discovery feeds and curated trusted sources, treating external data as actionable leads rather than verified reporting.

Scope & Standards

• Data Classification: Treats content from non-listed web sources as "Unverified Leads" per Journalistic Ethics Best Practices (SPOTLIGHT Guidelines).
• Search Behavior: All queries must return results from both sources but with distinct filtering capabilities.

Visual & Editorial Separation

• UI Indicator: Items originating from Open Net discovery must be tagged with a distinct UI marker (e.g., "Open Source" tag or color-coded border) to differentiate them from Trust-Led items.
• Labeling Protocol: The feed title for these items must explicitly include the classification text "Automated Discovery".

Data Attributes & Filtering

• Mandatory Metadata: Every discovery item must contain source (crawler or search engine name) and publication_time.
• User Filter: A global filter control must allow users to toggle between:
• Trusted Sources Only: Excludes all Open Net items.
• All Sources: Includes both curated feeds and open web leads.

Lead vs. Report Distinction

• Status Tag: A "Lead" status tag must be attached to any item not originating from the trusted allowlist (LL-012).
• Verification Requirement: No Open Net discovery item is marked as "Confirmed" or "Original Reporting" until it passes manual editorial review and enters a separate staging queue before publication.

LL-014: Editor Hide/Remove Toggle & Ingestion Log Specification

Objective Ensure that editors can suppress specific items from user view without deleting the underlying data, preserving the audit trail for compliance while maintaining content governance control.

Scope & Standards

• Security Standard: Adheres to NIST 800-53 Audit and Accountability controls.
• Data Integrity: Implements principles of non-destructive archival where original attribution cannot be altered or deleted by a user action.

Editor Action Interface

• Remove Toggle: A "Hide" button (instead of Delete) must be available on every item record in the editor dashboard.
• Reversibility: The "Hide" action must be reversible at any time via the global ingestion log or history interface.
• State Change: Changing from visible to hidden status updates only the user-facing feed query parameters; it does not alter the database record structure.

Ingestion Audit Log Requirements

• Immutable Record: Every hide or removal action must generate a timestamped audit entry that includes:
• Action: e.g., HIDDEN, RESTORED.
• User_ID: ID of the editor performing the action.
• Timestamp: ISO 8601 formatted UTC time.
• Target_ID: Unique identifier for the content item being managed.
• Provenance Preservation: The audit entry must explicitly note that original source attribution (Title, Source, Date) remains intact and immutable regardless of the hide status.

System Behavior

• Search Indexing: Items in "Hidden" state are marked with a metadata flag visibility: hidden but remain indexable for retrieval through the internal audit log interface.
• Notification: Editors must receive an immediate visual confirmation that the item has been logged and is no longer visible in the public feed.