Skip to main content

Notes and Agreements

As the OSCAL Foundation's FedRAMP Technology Focus Group (TFG) reaches agreements as to the path forward, they are captured here. This is a work in progress. Once the group reaches an appropriate milestone, this will be consolidated and re-organized into more consumable guidance.

WORK IN PROGRESS

Guiding Principles

We defined the following DRAFT guiding principles. Exceptions to these principles must be well-understood. The principles may be adjusted if warranted.

  • TARGET: KSI telemetry must support a clear and specific result, either:
    • a pass/fail determination; or
    • a consistently quantifiable "degree of health".
  • TIMING AND TRIGGERS: KSI telemetry must identify collection timing and/or triggers.
  • DATA QUALITY: KSI telemetry must distinguish direct data from derived or probabalistic data.

TARGET

All KSI telemetry is gathered in support of a clear and specific result.

Pass/Fail: Most compliance frameworks seek to define clear pass/fail criteria against specific controls or requirements, While pass/fail is still a preferred target for FedRAMP KSIs, it is not mandatory.

Degree of Health: FedRAMP KSIs open the door to other quantifiable metrics that can be interpreted to indicate a system's overall cybersecurity "health".

Telemetry reflecting "degree of halth" should include an unambiguous interpretation mechanism whenever practical, such as thresholds.

In some instances, measurements may require organization-specific or system-specific "baselining" before thresholds can be defined. This should be conducted within a defined period of time and adjusted periodically as necessary.

TIMING AND TRIGGERS

The intervals and/or triggers for KSI telemetry must be included in the KSI definition. KSI collection and delivery mechanisms must be aligned with this definition.

The FedRAMP PMO intends for CSPs to collect KSI telemetry via automation. Defined timing and triggers enable monitoring tools to know when to perform the collection and emit the telemetry. It also allows ingesting tools to know when to expect telemetry.

While defined intervals (frequency) is more predictable and preferred, some collection or telemetry cannot be anticipated and must be handled based on an event (trigger).

DATA QUALITY

Data Quality includes data integrity,

Iterpretations, Assertions and Probabalistic Data

  • Data:

    • Direct hard data (hard collection, hard correlation rules)
    • Human generated
    • Agentic AI generated

  • Assertions:

    • Logic-based correlation and thresholds
    • Agentic AI judgement
    • Human judgement

  • FRRs define the following roles relative to KSI definition:

    • CSP to define
    • Assessor to validate
    • PMO accecpts/rejects for certification
    • Agency accepts/rejects for authorization

  • KSI

  • Agentic AI requires identification of a responsible human.

    • In FedRAMP, defaults to system owner unless otherwise named.

Probabalistic vs Deterministic KSI Telemetry

KSI telemetry Whether because of Agentic AI interpretation or Although these terms are widely used in recent Agentic conversations

Agentic AI

  • PMO has stated a desire to lean into Agentic AI
  • There is a great deal of variance among Agencies on the acceptability of security telemetry, that has been generated or analyzed by Agentic AI.
  • There must be a mechanism for clarifying probabalistic telemetry from deterministic.

KSI Approach

  • KSI Statement Analysis

    • Identify KSI goal(s)

  • KSI Data Requirements:

    • Intent: A simple, unambiguous status. Typically Pass/Fail
    • Decision Point: Sampling or full coverage?
    • Identify evidence to collect in support of KSI goal(s)
    • Define evidence interpretation
      • Evidence type (count, true/false, setting)
      • Collection frequency
      • Evidence fidelity
      • Correlation
      • Thresholds
        • Could be more granular than just pass/fail
        • Example: Satisfactory, Degraded, Critical

  • Organizational Considerations:

    • Action Triggers: Define the triggers for that system/org
      • Define required action(s) when triggered

  • KSI Techical Collection Approach

    • Identify all evidence source/component
      • Identify the source format(s)
    • Define centralized evidence collection target
    • Define the automation required to acquire evidence from each source/component and deliver to centralied collection target
      • Decision Point: Evidence delivered raw or normalized?

  • KSI Technical Interpretation Approach

    • Apply data requirements/thresholds to produce Findings
      • Findings are continuously updating as new data is received and analyzed
    • Raise action triggers when appropriate
Back to top