Skip to main content

Notes and Agreements

As the OSCAL Foundation's FedRAMP Technology Focus Group (TFG) reaches agreements as to the path forward, they are captured here. This is a work in progress. Once the group reaches an appropriate milestone, this will be consolidated and re-organized into more consumable guidance.

WORK IN PROGRESS

Guiding Principles

We defined the following DRAFT guiding principles. Exceptions to these principles must be well-understood. The principles may be adjusted if warranted.

  • TARGET: Each KSI telemetrydefinition must supportdescribe aan clearintended andresult specificof result,data collection, either:
    • a pass/fail determination; or
    • a consistently quantifiable "degree of health".
  • RELEVANCE: Each defined KSI data stream must align to the KSI's defined target.
  • DATA ACQUISITION: Each defined KSI data stream must describe its acquisition mechanism , including the:
    • subject/source of the collected data
    • tool or automated agent performing the collection
  • DATA QUALITY: Each defined KSI data stream must distinguish base measures, derived measures, and probabalistic measures.
  • TIMING AND TRIGGERS: Each KSI telemetrydefinition must identify collection timing and/or triggers.triggers, this includes:
    • collection timing/triggers
    • DATAemition QUALITY:timing/triggers
      • KSI
    telemetry must distinguish direct data from derived or probabalistic data.

TARGET

All KSI telemetry is gathered in support of a clear and specific result.

Pass/Fail: Most compliance frameworks seek to define clear pass/fail criteria against specific controls or requirements, While pass/fail is still a preferred target for FedRAMP KSIs, it is not mandatory.

Degree of Health: FedRAMP KSIs open the door to other quantifiable metrics that can be interpreted to indicate a system's overall cybersecurity "health".

Telemetry reflecting "degree of halth" should include an unambiguous interpretation mechanism whenever practical, such as thresholds.

In some instances, measurements may require organization-specific or system-specific "baselining" before thresholds can be defined. This should be conducted within a defined period of time and adjusted periodically as necessary.

TIMING AND TRIGGERS

The intervals and/or triggers for KSI telemetry must be included in the KSI definition. KSI collection and delivery mechanisms must be aligned with this definition.

The FedRAMP PMO intends for CSPs to collect KSI telemetry via automation. Defined timing and triggers enable monitoring tools to know when to perform the collection and emit the telemetry. It also allows ingesting tools to know when to expect telemetry.

While defined intervals (frequency) is more predictable and preferred, some collection or telemetry cannot be anticipated and must be handled based on an event (trigger).

DATA QUALITY

DataISO/IEC Quality15939 (software measurement process), includeswhich data integrity,

Iterpretations, Assertions and Probabalistic Data

  • Data:defines:

    • DirectBase hardmeasure data— directly observed, single source, no computation (hardyour collection,"direct hard correlation rules)read")
    • HumanDerived generatedmeasure — function of two or more base measures (your "calculation")
    • AgenticIndicator AI— generateda derived measure used to support a decision (all KSI drived measures in used to support a satisfaction or health decision)

  • Assertions:

    • Logic-based correlation and thresholds

    • Agentic AI judgement

    • Human judgement

  • FRRs define the following roles relative to KSI definition:

    • CSP to define
    • Assessor to validate
    • PMO accecpts/rejects for certification
    • Agency accepts/rejects for authorization

  • KSI

  • Agentic AI requires identification of a responsible human.

    • In FedRAMP, defaults to system owner unless otherwise named.

Probabalistic vs Deterministic KSI Telemetry

KSI telemetry Whether because of Agentic AI interpretation or Although these terms are widely used in recent Agentic conversations

Agentic AI

  • PMO has stated a desire to lean into Agentic AI
  • There is a great deal of variance among Agencies on the acceptability of security telemetry, that has been generated or analyzed by Agentic AI.
  • There must be a mechanism for clarifying probabalisticAgentic AI derived telemetry from deterministic.more deterministic measures.

KSI Approach

  • KSI Statement Analysis

    • Identify KSI goal(s)

  • KSI Data Requirements:

    • Intent: A simple, unambiguous status. Typically Pass/Fail
    • Decision Point: Sampling or full coverage?
    • Identify evidence to collect in support of KSI goal(s)
    • Define evidence interpretation
      • Evidence type (count, true/false, setting)
      • Collection frequency
      • Evidence fidelity
      • Correlation
      • Thresholds
        • Could be more granular than just pass/fail
        • Example: Satisfactory, Degraded, Critical

  • Organizational Considerations:

    • Action Triggers: Define the triggers for that system/org
      • Define required action(s) when triggered

  • KSI Techical Collection Approach

    • Identify all evidence source/component
      • Identify the source format(s)
    • Define centralized evidence collection target
    • Define the automation required to acquire evidence from each source/component and deliver to centralied collection target
      • Decision Point: Evidence delivered raw or normalized?

  • KSI Technical Interpretation Approach

    • Apply data requirements/thresholds to produce Findings
      • Findings are continuously updating as new data is received and analyzed
    • Raise action triggers when appropriate
Back to top