Article last updated in July 2022 for CCSS Version 8.0.
Marc Krisjanous is one of the first CCSS Auditors and assisted C4 in the development of their auditors program.
In this series we will review each of the core Aspects in the CCSS and provide our interpretation for each of the Aspect’s requirements and what possible evidence could provide assurance to the auditor that a requirement is in-place. Make sure to read our other in-depth articles on the CCSS Aspects:
CCSS Aspect 2.01 Security Tests / Audits
In this article we will explore how an auditor could interpret the CCSS Aspect 2.01 Security Tests/ Audits.
Aspect 2.01 Security Tests/ Audits addresses the importance of an independent (third-party) review of the in-scope people, process, and technology. The Aspect defines vulnerability scans, penetration tests and security audits. The Aspect’s objective defined within the CCSS is provided below.
“This aspect covers third-party reviews of the security systems, technical controls, and policies that protect the information system from all forms of risk as well as vulnerability and penetration tests designed to identify paths around existing controls. Regardless of the technical skills, knowledge, and experience of staff who build and maintain an information system, it has been proven that third-person reviews often identify risks and control deficiencies that were either overlooked or underestimated by staff. For the same reasons that development companies require different people to test a product from those who write it, different people than those who implement a cryptocurrency system should assess its security. Third parties provide a different viewpoint and are independent of the technical controls and can be objective without risk of retaliation.“
Vulnerability Scanning and Pen-tests – there is a difference
It’s important to note the key differences between a vulnerability scan and a penetration test (we go into this more in one of our other posts on penetration testing versus vulnerability scanning).
A vulnerability scan is a process where systems are scanned for vulnerabilities. The scan can be in-depth in which the scanner has login credentials to servers/devices and systems – scanning from the operating system level and up the software stack. This type of in-depth scan is often referred to as an “internal vulnerability scan” as the scanner is scanning internal network servers, devices, and systems.
Another type of scan is an “external vulnerability scan” which just scans the publicly accessible interfaces of a system, for example a web application’s public web pages. The external scanner may also have login credentials and log into the web application to scan more web pages.
A penetration test or “pen-test” comprises firstly of a vulnerability scan to identify vulnerabilities in the target systems. Pen-tests almost always start with a vulnerability scan to detect vulnerabilities. For some penetration test engagements, the assessed entity may provide a list of vulnerabilities to attack, or the penetration testers are aware of the vulnerabilities inherent to the systems already. Once the vulnerabilities identified are reviewed by the pen-testers who then decide what vulnerabilities to attack to gain unauthorized access to the target systems. If the attack is successful and the pen-tester gains unauthorized access to the target systems then the pen-tester, if instructed by the client, will attempt to see how much further they can get into the systems by targeting other vulnerabilities or weaknesses in the system that could not have been exploited initially.
Code Review/Audit
The review of code for vulnerabilities can be conducted using tools that analyze the code or are undertaken by a code reviewer who is skilled in the coding language. Often both approaches are utilized and not only are industry best practices referenced as a guide during the review process, but the assessed entities coding standards are referenced to ensure the standards have been adhered to. CCSS also requires that the review/audit include a review of the cryptographic libraries used to ensure they have been implemented correctly.
CCSS Levels
CCSS provides three levels of compliance – Level 1 being the base level of implementing CCSS requirements up to Level 3 being the most in-depth implementation of CCSS requirements. We shall review each compliance level and provide our thoughts on what evidence an auditor should seek to provide assurance that the requirements are in-place.
CCSS Level 1 Requirements
CCSS Level 1 requirements for this Aspect is provided below.
Our interpretation of this Level 1 requirement is that the auditor is seeking evidence that a developer skilled in the cryptocurrency protocol being implemented assists in the design and implementation of the system using the cryptocurrency. Evidence that provides assurance this requirement is in-place could be:
- The developer(s) roles and responsibilities for the project are documented and approved by senior management who have accountability for the project.
- Evidence of the developer’s skill and experience in the cryptocurrency has been reviewed by the relevant project members or other resources to ensure the developer is skilled enough for the role. Evidence provided could be certifications for the development, design and or implementation of the cryptocurrency or other relevant technologies. Other company references for the developer could be included as well.
- We would also recommend that the auditor seeks some form of evidence that the developer has a good understanding of secure coding techniques – preferably the developer has undertaken secure coding training relevant to the coding language and platform. The developer may have the required skill for the cryptocurrency being used but if they do not know and apply the basics of secure coding techniques then this is a major risk to the project. For example, hard coding a private key into the source code is not a good idea but it does happen.
CCSS Level 2 Requirements
CCSS Level 2 requirements for this Aspect are provided below.
As mentioned above, there is a key difference between a vulnerability scan and a penetration test. To put it another way – a vulnerability scan will detect “holes” in the system. Where a penetration test will detect the “holes” and use one or more holes to gain access into the systems behind and see what other vulnerabilities are around. The goal of most penetration tests is to see if the pen-tester can exploit the vulnerabilities to a level where they have complete control of all systems in-scope.
Level 2 requirements state that regular security assessments are conducted which include vulnerability scanning and pen-testing. Breaking this requirement down into more manageable pieces for audit we have identified the following audit considerations:
- Level 2 requires a regular security assessment that includes a vulnerability scan and pen-test. The CCSS Committee has not provided a definition of “regular”. Is it every month? Every quarter? Every year? It is our opinion that only the assessed entity can define the frequency based on several factors including: (1) the relevant risks identified by the assessed entity for their cryptocurrency products and services, (2) the amount of change and the complexity of the change for systems that provide cryptocurrency functions. For example, changing software coding language and having to rewrite many components should trigger a security assessment regardless if a security assessment was completed a month before, (3) regulatory or statutory requirements.
- Depending on factors such as the scope of the testing, the skill of the pen-testers, time given to test the in-scope environment, and the configuration and hardening applied to the systems – penetration testing and vulnerability scan findings may be considerable requiring a great deal of time to remediate. The findings might be so severe as to cause the entire system to be removed from production until fixed.
- Level 2 requires that the auditing and testing tasks are conducted by a third-party entity. There is currently no definition provided by the CCSS Committee for “third party entity” so let us refer to the International Organization for Standardization (ISO) terms of reference database. ISO provides a couple of definitions for “third party” depending on the standard the term is applied too. A definition that is applicable to our needs is “person or body that is recognized as being independent of the parties involved with respect to the issues in question“. The key word in this definition is “independent“. This definition is still too vague for most audits as “independent” could be defined as a person or team who is employed by the same organization that is building and implementing the system but not directly involved with that project. Is that interpretation acceptable for you as an auditor or would you require an entity outside of the in-scope organization? As an auditor you may have to discuss with the assessed entity and reach an agreement on what “third party entity” means. Not to go on, but another consideration is what type of penetration testing was conducted? Was it, Internal? External? Segmentation? Blackbox? Whitebox? Grey Box?
- On the subject of code auditing, vulnerability scanning and penetration testing – CCSS does not provide any guidance as to the professionalism or certification requirements of the third party who will conduct these tasks. As an example, the assessed entities CEO’s 20-year-old son who dabbles in “hacking” using Kali Linux every night in his room, in our opinion, would not be recognized by Confide‘s auditors as being a professional penetration tester who holds formal qualifications in penetration testing, works for a company that is Crest certified (for example) and uses an industry recognized standards-based approach for penetration testing such as NIST SP800-115.
- There is also a requirement for remediation evidence: “Documentation shows that all concerns raised by the assessment have been evaluated for risk and addressed by the organization.“. Firstly, we will assume that by “assessment” CCSS means the findings from the vulnerability scan or the penetration test. The findings should be provided as official documentation to the auditor, meaning the report authored by the third-party conducting the audit, with all findings listed. The auditor should ensure that the report has not been edited as to hide any findings from the audit. As part of the evidence, a remediation list should also be provided to the auditor which outlines the assessed entities assessment of each finding as to whether they believe the finding to be a false positive or a valid finding, what is the remediation to address the issue, the date the finding will be addressed (e.g fixed) and how will the remediation be checked to show that each finding has been successfully addressed. CCSS does not define if another assessment in full or in-part, vulnerability scan or penetration test needs to be conducted once all accepted findings have been addressed to confirm all findings have been sufficiently remediated. This leaves the auditor with the challenge as to how they can confirm the remediation has been completed successfully.
CCSS Level 3 Requirements
CCSS Level 3 requirements for this aspect is provided below.
The Level 3 requirements provide a subtle change in the wording from Level 2. Note that for Level 2 compliance a security assessment is required but for Level 3 compliance an audit is required.
What’s the difference between an assessment and an audit?
In our opinion, an audit is a process conducted by a third-party who is outside of the organization that reviews evidence collected from the entity being audited and verifies their compliance to a standard or policy. While there are audits that are undertaken by internal auditors (called an internal audits),CCSS requires a third-party audit, which we have defined as completely independent of the assessed entity.
An assessment process reviews the evidence provided and reaches a conclusion regarding the state of the entity at that point in time, in readiness for compliance to a standard or policy. While PCI DSS uses the term “assessment” it is more akin to the traditional meaning of audit as it is an external third-party audit to validate compliance against a standard. For the purposes of this article, we are not talking about this type of assessment.
For example, you may have heard of a “readiness assessment for audit”. This is where a person(s) internal or external to the organization, can review the environment that will be audited and determine what will be compliant and what will not. Then a remediation plan is generally created so the organization can implement changes to reach compliance.
CCSS for Level 3 provides a required frequency for audits which is at a minimum on a yearly basis. It should be noted that CCSS does not require an audit to be conducted if there are any significant changes to the CCSS in-scope environment.
Summary
In this post we explored Aspect 2.01 Security Tests/ Audits with our auditor’s hat on to determine how an auditor could approach the evidence gathering required so that an opinion could be reached as to if the assessed entity is compliant to Level 1, Level 2 or Level 3 for this Aspect.
We discovered that many of the Aspect’s requirements required a reasonable amount of interpretation which adds risk that the assessed entity may reach a different interpretation than the auditor which is never a good situation to be in for either party.
Ultimately, it is the auditor who risks their standing, auditing license and auditing career by signing a certification of compliance for the assessed entity. The auditor must be confident that the requirements have been meet and that the standard provides enough detail and clarity that both the assessed entity and the auditor can reach the same conclusion as to what evidence is required to prove that the requirement is in-place.
CCSS is still a “young” information security management systems standard however, the standards objectives are very clear, and we believe CCSS will improve and mature overtime and be a highly respected standard to be compliant in.
