Skip to main content

Notes and Agreements

As the OSCAL Foundation's FedRAMP Technology Focus Group (TFG) reaches agreements as to the path forward, they are captured here. This is a work in progress. Once the group reaches an appropriate milestone, this will be consolidated and re-organized into more consumable guidance.

WORK IN PROGRESS

Guiding Principles

WenWe planningdefined KSIthe telemetry,following it:DRAFT guiding principles. Exceptions to these principles must be well-understood. The principles may be adjusted if warranted.

  • TARGET: KSI telemetry must support a clear and specific result, either:
    • a pass/fail determinationdetermination; or
    • a quatifiable and consistently appliedquantifiable "degree of health".
  • mustTIMING beAND quantative
    • TRIGGERS:
  • KSI telemetry must identify collection timing and/or triggerstriggers.
  • DATA QUALITY: KSI telemetry must distinguish direct data from derived or probabalistic data.

TARGET

All KSI telemetry is gathered in support of a clear and specific result.

Pass/Fail: Most compliance frameworks seek to define clear pass/fail criteria against specific controls or requirements, While pass/fail is still a preferred target for FedRAMP KSIs, it is not mandatory.

Degree of Health: FedRAMP KSIs open the door to other quantifiable metrics that can be interpreted to indicate a system's overall cybersecurity "health".

Telemetry reflecting "degree of halth" should clarifyinclude fidelityan unambiguous interpretation mechanism whenever practical, such as thresholds for degree of health. In some instances, measurements may require organization-specific or system-specific "baselining" before thresholds can be defined. This should be conducted within a defined period of time and adjusted periodically as necessary.

TIMING AND TRIGGERS

<<<<====----The WORKINGFedRAMP HEREPMO intends CSPs to collect KSI telemetry via automation. The frequency and/or timing of automated collection must be included in the KSI definition and aligned with the KSI collection mechanisms.

DATA QUALITY

Iterpretations, Assertions and Probabalistic Data

  • WeData:

    may
      find
    • Direct thishard isn'tdata always(hard prossible.collection, Exceptionshard mustcorrelation berules)
      • well-justified.

    • Human generated
    • Agentic AI generated

  • Measurements may be other quantifiable data, such as % based, nouns, countsAssertions:

    • QuantifiableLogic-based datacorrelation must haveand thresholds defined
    • thresholdsAgentic mustAI tranlsatejudgement
      • measurements
    • Human to pass/failjudgement

  • FRRs define the following roles relative to KSI definition:

    • CSP to define
    • Assessor to validate
    • PMO accecpts/rejects for certification
    • Agency accepts/rejects for authorization

  • Need to be able to distinguish:

    • Data:
      • Direct hard data (hard collection, hard correlation rules)
      • Human generated
      • Agentic AI generated
    • Assertions:
      • Logic-based correlation and thresholds
      • Agentic AI judgement
      • Human judgement

  • KSI

  • Agentic AI requires identification of a responsible human.

    • In FedRAMP, defaults to system owner unless otherwise named.

Probabalistic vs Deterministic KSI Telemetry

KSI telemetry Whether because of Agentic AI interpretation or Although these terms are widely used in recent Agentic conversations

Agentic AI

  • PMO has stated a desire to lean into Agentic AI
  • There is a great deal of variance among Agencies on the acceptability of security telemetry, that has been generated or analyzed by Agentic AI.
  • There must be a mechanism for clarifying probabalistic telemetry from deterministic.

KSI Approach

  • KSI Statement Analysis

    • Identify KSI goal(s)

  • KSI Data Requirements:

    • Intent: A simple, unambiguous status. Typically Pass/Fail
    • Decision Point: Sampling or full coverage?
    • Identify evidence to collect in support of KSI goal(s)
    • Define evidence interpretation
      • Evidence type (count, true/false, setting)
      • Collection frequency
      • Evidence fidelity
      • Correlation
      • Thresholds
        • Could be more granular than just pass/fail
        • Example: Satisfactory, Degraded, Critical

  • Organizational Considerations:

    • Action Triggers: Define the triggers for that system/org
      • Define required action(s) when triggered

  • KSI Techical Collection Approach

    • Identify all evidence source/component
      • Identify the source format(s)
    • Define centralized evidence collection target
    • Define the automation required to acquire evidence from each source/component and deliver to centralied collection target
      • Decision Point: Evidence delivered raw or normalized?

  • KSI Technical Interpretation Approach

    • Apply data requirements/thresholds to produce Findings
      • Findings are continuously updating as new data is received and analyzed
    • Raise action triggers when appropriate
Back to top