Skip to main content

Notes and Agreements

As the OSCAL Foundation's FedRAMP Technology Focus Group (TFG) reaches agreements as to the path forward, they are captured here. This is a work in progress. Once the group reaches an appropriate milestone, this will be consolidated and re-organized into more consumable guidance.

WORK IN

DRAFT PROGRESS- Please review and provide feedback. Self-register via Login to leave comments.

Taxonomy

We defined the following DRAFT KSI Taxonomy to align with CR26 terminology and Security Decision Record (SDR) schema fields. We expanded on this where necessary, but avoided deviations.

Taxonomy elements are named in the singular; a KSI's Design may reference more than one instance of any element (e.g., multiple Data Sources reconciled by a single Data InterpretationAnalysis rubric).

The taxonomy of a KSI includes:

  • Requirement: The required KSI as defined by the PMO in CR26.
  • DesignDefinition (What): Explanation of measures and objectives demonstrating KSIs. Consistent with SDR-CSX-KSI, this content is provided by the CSP in the SDR. The designdefinition includes:
    • Subject: The capability the KSI's claim is about.
    • Attribute: The specific observable property of that subjectSubject being examined.
    • Cycle: The timing pattern governing when data is collected. Expressed either as a frequency or as afrequency, trigger or both.
    • Rationale: Why the measurement supports the KSI.
  • Acquisition and Analysis (How):
    • Data Source: Where evidenceEvidence of an attribute is found.
      • Acquisition Basis: Categorizes the nature of the dataEvidence as received, before Data Analysis is applied.performed. See AnalysisMeasurement Basis below for details.
      • Provenance Agent: Categorizes the mechanism that makes the data available ahead of its acquisition by the Collection Mechanism. See Agent Basis below for details.
    • RationaleData Characteristics: WhyThe format, units of other characteristics of the measurementEvidence supportsthat theimpact KSI.Data Analysis.
    • Data Analysis: The correlation logic, thresholds, or other rubric applied to Telemetry to produce a Finding. See Measurement Basis below for details.
      • Analysis Basis: Categorizes the nature of the logic applied to produce the Finding.
    • DataAnalysis TypesAgent: TheCategorizes formatthe andmechanism unitsthat ofperforms collectedthe data.analysis. See Agent Basis below for details.
  • Collection Mechanism: A description of an automated mechanism performing data acquisition, including the technical method used to acquire the evidence, such as an API call.
  • Analysis Mechanism: A description of the mechanism performing Data Analysis.
  • Evidence:
    • Telemetry: The actual data acquired by a Collection Mechanism from its Data Source, per the defined Cycle.
    • Finding: The conclusion reached by applying the Data Analysis to the actual Telemetry. This is the KSI-aligned determination of the subject's performance.

    • Guiding Principles

    We defined the following DRAFT guiding principles.

    • Data Analysis: The Data Analysis must clearly define the intended result of data collection, either:
      • a pass/fail determination; or
      • a consistently quantifiable degree of health.
    • Measurement Basis: Data Source and Data Analysis must distinguish Base, Derived, Interpretive and Probabilistic Measures, including those measures produced by Agentic AI.
    • Agent Basis: The Telemetry and Findings must identify the involvement of Automation, Statistical Model, Human and/or Agentic AI.

    Data Analysis

    The Data Analysis must clearly define the intended result of data collection, either:

    • a Pass/Fail determination; or
    • a consistently quantifiable Degree of Health.

    All KSI telemetry is gathered in support of a clear and specific result.

    Pass/Fail

    Most compliance frameworks seek to define clear pass/fail criteria against specific controls or requirements.

    While Pass/Fail is still a preferred target for FedRAMP KSIs, it is not mandatory.

    Degree of Health

    FedRAMP KSIs open the door to other quantifiable metrics that can be analyzed to indicate a system's overall cybersecurity "health". Telemetry reflecting Degree of Health should include an unambiguous analysis mechanism whenever practical, such as thresholds.

    In some instances, measurements may require organization-specific or system-specific "baselining" before thresholds can be defined. This should be conducted within a defined period of time and adjusted periodically as necessary.

    Measurement Basis

    Data Source and Data Analysis must distinguish Base, Derived, Interpretive and Probabilistic Measures, including those measures produced by Agentic AI.

    Categories

    The Measurement Basis Categories are:

    • Concrete Measures: Are quantitative without subjectivity.
      • Base Measure: A measure obtained by direct observation of a single attribute, requiring no computation or inference from other measures. (ISO 15939 term)
      • Derived Measure: A measure computed as a function of two or more Base and/or Derived Measures. (ISO 15939 term)
    • Inferential Measures: May be quantitative or qualitative and have some element of subjectivity.
      • Interpretive Measure: A measure produced through human or Agentic AI judgment rather than deterministic computation or statistical inference — a qualitative assessment that cannot be fully reduced to a formula.
      • Probabilistic Measure: A measure produced by applying a statistical model, threshold, or algorithmic inference to underlying data, yielding a likelihood, score, or classification rather than a directly observed value.

    Application

    Measurement Basis applies to both Acquisition Basis and Analysis Basis.

    • Acquisition Basis: The Measurement Basis classification of the data as received from the Data Source, before the KSI's own Data InterpretationAnalysis is applied.
    • Analysis Basis: The Measurement Basis classification of the logic the KSI itself applies to produce the Finding.

    A Finding's Acquisition Basis and an InterpretationAnalysis Basis may differ from one another. For example, A Finding built on a rigorously computed DerivedConcrete Measure is still, in effect, aan ProbabilisticInferential Finding if the underlying Data Source itself returned ana Probabilistic value.

    Both Acquisition Basis and InterpretationAnalysis Basis must be provided such that aan OrganizationsAgency can filter based on their risk tolerance for interpretiveEvidence orbased deterministicon evidence.Inferential Measures.

    Agentic

    Agent AI

    Basis

    Although Agentic AI is implicitly covered by Interpretive and Probabilistic Measures, Agencies differ in their acceptance of Agentic AI usage. For this reason,The Telemetry and Findings should furthermust identify datathe generatedinvolvement of Automation, Statistical Model, Human and/or Agentic AI.

    Categories

    The Agent Basis Categories are:

    • Automation: Deterministic, rule-based, automated mechanism.
    • Statistical Model: Machine Learning, probabilistic, non-agentic mechanism.
    • Human: Manual assembly, subjective analysis, judgement calls or similar activities performed by anhumans.
    • Agentic AI:

    Agencies need a way to filter Evidence based on strictly deterministic automation or less deterministic influences from humans, machine-learning and Agentic AI,AI allowing Agencies to accept or filter this data.mechanisms.

    KSI Approach

    • KSI Statement Analysis

      • Identify KSI goal(s)

    • KSI Data Requirements:

      • Intent: A simple, unambiguous status. Typically Pass/Fail
      • Decision Point: Sampling or full coverage?
      • Identify evidence to collect in support of KSI goal(s)
      • Define evidence interpretation
        • Evidence type (count, true/false, setting)
        • Collection frequency
        • Evidence fidelity
        • Correlation
        • Thresholds
          • Could be more granular than just pass/fail
          • Example: Satisfactory, Degraded, Critical

    • Organizational Considerations:

      • Action Triggers: Define the triggers for that system/org
        • Define required action(s) when triggered

    • KSI Techical Collection Approach

      • Identify all evidence source/component
        • Identify the source format(s)
      • Define centralized evidence collection target
      • Define the automation required to acquire evidence from each source/component and deliver to centralied collection target
        • Decision Point: Evidence delivered raw or normalized?

    • KSI Technical Interpretation Approach

      • Apply data requirements/thresholds to produce Findings
        • Findings are continuously updating as new data is received and analyzed
      • Raise action triggers when appropriate
    Back to top