Notes and Agreements

As the OSCAL Foundation's FedRAMP Technology Focus Group (TFG) reaches agreements as to the path forward, they are captured here. This is a work in progress. Once the group reaches an appropriate milestone, this will be consolidated and re-organized into more consumable guidance.

Taxonomy

We defined the following DRAFT KSI Taxonomy to align with CR26 terminology and Security Decision Record (SDR) schema fields. We expanded on this where necessary, but avoided deviations.

Taxonomy elements are named in the singular; a KSI's Design may reference more than one instance of any element (e.g., multiple Data Sources reconciled by a single Data Interpretation rubric).

The taxonomy of a KSI includes:

• Requirement: The required KSI as defined by the PMO in CR26.
• Design: Explanation of measures and objectives demonstrating KSIs. Consistent with SDR-CSX-KSI, this content is provided by the CSP in the SDR. The design includes:
  • Subject: The capability the KSI's claim is about.
  • Attribute: The specific observable property of that subject being examined.
  • Cycle: The timing pattern governing when data is collected. Expressed either as a frequency or as a trigger or both.
  • Data Source: Identifies where evidence of that attribute is found.
    • Acquisition Basis: Classifies the nature of the data as received, before the KSI's own Data Interpretation is applied. See Measurement Basis below for details.
  • Rationale: Why the measurement supports the KSI.
  • Data Interpretation: The correlation logic, thresholds, or other rubric applied to Telemetry to produce a Finding. See Measurement Basis below for details.
    • Interpretation Basis: Classifies the nature of the logic the KSI itself applies to produce the Finding.
  • Data Types: The format and units of collected data.
• Collection Mechanism: A description of an automated mechanism performing data acquisition, including the technical method used to acquiring the evidence, such as an API call.
• Evidence:
  • Telemetry: The actual data acquired by a Collection Mechanism from its Data Source, per the defined Cycles.
  • Finding: The conclusion reached by applying the Design's Data Interpretation rubric to the actual Telemetry. This is the KSI-aligned determination of the subject's performance.

Guiding Principles

We defined the following DRAFT guiding principles.

• Subjects: Each KSI must identify at least one assessment ojbect, and may define multiple subjects.
• Data Interpretation: The Data Interpretation must clearly deifne the intended result of data collection, either:
  • a pass/fail determination; or
  • a consistently quantifiable degree of health.
• Measurement Basis: Each defined KSI data source must distinguish base measures, derived measures, interpretive and probabilistic measures.

Measurement Basis

• Base measure: A measure obtained by direct observation of a single attribute, requiring no computation or inference from other measures. (ISO 15939 term)
• Derived measure: A measure computed as a function of two or more Base and/or Derived Measures. (ISO 15939 term)
• Interpretive Measure: A measure produced through human or Agentic AI judgment rather than deterministic computation or statistical inference — a qualitative assessment that cannot be fully reduced to a formula.
• Probabilistic Measure: A measure produced by applying a statistical model, threshold, or algorithmic inference to underlying data, yielding a likelihood, score, or classification rather than a directly observed value.

Data Interpretation

All KSI telemetry is gathered in support of a clear and specific result.

Pass/Fail: Most compliance frameworks seek to define clear pass/fail criteria against specific controls or requirements, While pass/fail is still a preferred target for FedRAMP KSIs, it is not mandatory.

Degree of Health: FedRAMP KSIs open the door to other quantifiable metrics that can be interpreted to indicate a system's overall cybersecurity "health".

Telemetry reflecting "degree of halth" should include an unambiguous interpretation mechanism whenever practical, such as thresholds.

In some instances, measurements may require organization-specific or system-specific "baselining" before thresholds can be defined. This should be conducted within a defined period of time and adjusted periodically as necessary.

TIMING AND TRIGGERS

The intervals and/or triggers for KSI telemetry must be included in the KSI definition. KSI collection and delivery mechanisms must be aligned with this definition.

The FedRAMP PMO intends for CSPs to collect KSI telemetry via automation. Defined timing and triggers enable monitoring tools to know when to perform the collection and emit the telemetry. It also allows ingesting tools to know when to expect telemetry.

While defined intervals (frequency) is more predictable and preferred, some collection or telemetry cannot be anticipated and must be handled based on an event (trigger).

Agentic AI

• PMO has stated a desire to lean into Agentic AI
• There is a great deal of variance among Agencies on the acceptability of security telemetry, that has been generated or analyzed by Agentic AI.
• There must be a mechanism for clarifying Agentic AI derived telemetry from more deterministic measures.

KSI Approach

• KSI Statement Analysis

  • Identify KSI goal(s)

• KSI Data Requirements:

  • Intent: A simple, unambiguous status. Typically Pass/Fail
  • Decision Point: Sampling or full coverage?
  • Identify evidence to collect in support of KSI goal(s)
  • Define evidence interpretation
    • Evidence type (count, true/false, setting)
    • Collection frequency
    • Evidence fidelity
    • Correlation
    • Thresholds
      • Could be more granular than just pass/fail
      • Example: Satisfactory, Degraded, Critical

• Organizational Considerations:

  • Action Triggers: Define the triggers for that system/org
    • Define required action(s) when triggered

• KSI Techical Collection Approach

  • Identify all evidence source/component
    • Identify the source format(s)
  • Define centralized evidence collection target
  • Define the automation required to acquire evidence from each source/component and deliver to centralied collection target
    • Decision Point: Evidence delivered raw or normalized?

• KSI Technical Interpretation Approach

  • Apply data requirements/thresholds to produce Findings
    • Findings are continuously updating as new data is received and analyzed
  • Raise action triggers when appropriate