Skip to main content

Notes and Agreements

As the OSCAL Foundation's FedRAMP Technology Focus Group (TFG) reaches agreements as to the path forward, they are captured here. This is a work in progress. Once the group reaches an appropriate milestone, this will be consolidated and re-organized into more consumable guidance.

WORK IN PROGRESS

Guiding Principles

Wen planning KSI telemetry, it:

  • must support either:
    • a pass/fail determination
    • a quatifiable and consistently applied "degree of health"
  • must be quantative
  • must identify collection timing and/or triggers
  • should clarify fidelity

<<<<====---- WORKING HERE

  • We may find this isn't always prossible. Exceptions must be well-justified.

  • Measurements may be other quantifiable data, such as % based, nouns, counts

    • Quantifiable data must have thresholds defined
    • thresholds must tranlsate measurements to pass/fail

  • FRRs define the following roles relative to KSI definition:

    • CSP to define
    • Assessor to validate
    • PMO accecpts/rejects for certification
    • Agency accepts/rejects for authorization

  • Need to be able to distinguish:

    • Data:
      • Direct hard data (hard collection, hard correlation rules)
      • Human generated
      • Agentic AI generated
    • Assertions:
      • Logic-based correlation and thresholds
      • Agentic AI judgement
      • Human judgement

  • KSI

  • Agentic AI requires identification of a responsible human.

    • In FedRAMP, defaults to system owner unless otherwise named.

Probabalistic vs Deterministic KSI Telemetry

KSI telemetry Whether because of Agentic AI interpretation or Although these terms are widely used in recent Agentic conversations

Agentic AI

  • PMO has stated a desire to lean into Agentic AI
  • There is a great deal of variance among Agencies on the acceptability of security telemetry, that has been generated or analyzed by Agentic AI.
  • There must be a mechanism for clarifying probabalistic telemetry from deterministic.

KSI Approach

  • KSI Statement Analysis

    • Identify KSI goal(s)

  • KSI Data Requirements:

    • Intent: A simple, unambiguous status. Typically Pass/Fail
    • Decision Point: Sampling or full coverage?
    • Identify evidence to collect in support of KSI goal(s)
    • Define evidence interpretation
      • Evidence type (count, true/false, setting)
      • Collection frequency
      • Evidence fidelity
      • Correlation
      • Thresholds
        • Could be more granular than just pass/fail
        • Example: Satisfactory, Degraded, Critical

  • Organizational Considerations:

    • Action Triggers: Define the triggers for that system/org
      • Define required action(s) when triggered

  • KSI Techical Collection Approach

    • Identify all evidence source/component
      • Identify the source format(s)
    • Define centralized evidence collection target
    • Define the automation required to acquire evidence from each source/component and deliver to centralied collection target
      • Decision Point: Evidence delivered raw or normalized?

  • KSI Technical Interpretation Approach

    • Apply data requirements/thresholds to produce Findings
      • Findings are continuously updating as new data is received and analyzed
    • Raise action triggers when appropriate
Back to top