G-Cloud 14: What's stopping you? PART TWO

Executive Summary

• Part 2 of a 2-Part article on SMEs failing to sell on G-Cloud
• Being admitted to G-Cloud does not mean your details are compliant with the rules, there is no Quality Control
• Summary of Part 1
• The Importance of Filters in search
• Pricing is a minefield, 50% of SMEs fail hard, here
• Service Definition, 37% of SMEs fail on this one
• Accessibility, 70% of SMEs respond “none or don’t know”
• Cyber Security Standards
• Terms & Conditions
• Overall Quality & Standard of Presentation
• Conclusion – most problems can be fixed, now is the time!

Introduction

In Part 1, I started looking for the answer to the question: For SMEs, with viable products but without any sales success on the G-Cloud frameworks, “What’s stopping you?”

It has been a feature of G-Cloud since its inception in 2012, indeed it was one of its founding principles, that SMEs can be very successful selling to the public sector through this framework. It has also been very evident from the beginning that more SMEs fail ever to make a single sale than get traction. The proportion of SMEs with some sales out of the total population of all SMEs on the framework at any time has always been around 25%-30%. (It will be smaller immediately after a new framework is launched).

Crown Commercial Service (“CCS”) do not perform a quality control function. Admission of a supplier to a framework does not certify that the supplier has all the required details properly presented on the Digital Marketplace. In many cases they do not and in many cases these errors are sufficiently bad to prevent any buyer from considering purchasing a service.

In other cases, the errors can limit a supplier’s potential market.

In Part 1, I set out the methodology I had adopted to look at a 5% sample of unsuccessful UK registered SMEs with a Cloud Software offering (but ignoring resellers). I then read the listing particulars for one Software service for each of the sample of 94 suppliers as well as those of a Control Group of 10 successful SMEs on Lot 2.

My subjective review of the 94 identified that 77% of them had one or more errors that in my opinion were so severe that a buyer could not legitimately enter into a transaction with them.

I then started looking at the main types of error, identifying first some suppliers that should not be on the framework and secondly that the majority of unsuccessful SMEs did not understand how a buyer identifies services to evaluate through the search facility.

Part 2 – Concluding the analysis

In this second part of the article, I will concentrate on the following areas which I have found to have a high potential for fatal or seriously limiting mistakes and misunderstandings:

• Filters
• Pricing
• Service Definition
• Accessibility
• Standards (e.g. ISO27001, Cyber Essentials)
• Terms & Conditions
• Overall quality and standard of presentation

The importance of Filters in search

The G-Cloud Buyers’ Guide instructs buyers to:

“Choose a category, then search for services using keywords and filters. Save your search so you can export your results later.”

And…

“What to do if you have too many results…

…You must assess all of the services in your search results. If you have too many services to assess, add filters to refine your results. The filters you add should be based on your requirements.”

With 40,000 services on G-Cloud, understanding and optimising your service for Filters is as important as Search optimisation. There was a large variation in the degree to which suppliers in the sample seemed prepared to go the distance to ‘tick the extra boxes’ necessary to be optimised for Filters. This could be severely limiting as buyers, confronted with many services to evaluate, will probably look negatively on a supplier that for “Staff Security Clearance” responds “Staff screening not performed”.

Although a subjective measure, after reviewing all the detail of a service, I have allocated a score based on my impression of the supplier’s awareness and efforts to move off the baseline level of compliance with some of the Filter fields:

Table comparing effort to optimising for Filter Fields

This would appear to indicate a rich seam of opportunity for improvement. The Control Group, which has been selected to be a reflection of the reasonably successful SMEs, neither shooting stars nor ‘toddlers’, are clearly into the ‘tick-box’ awareness zone. They do not want to be excluded from a search for something relatively cheap to embrace, which may also improve the quality of the service they deliver for all their customers, both public and private sector.

Pricing

Pricing is a minefield. In my opinion, 50% of the sample of SMEs reviewed had submitted pricing information that was incomplete, incoherent or too vague for a buyer to evaluate the cost of using the service. This should be a ‘fatal error’ stopping the buyer from considering the service further.

Even in the ‘Control Group’ of 10 successful SMEs, two of the sample stated in their pricing document that pricing would increase annually either by reference to an inflation index or a stated annual percentage. This is not permitted (see ‘Clarification statements’ below). It is difficult to see how this could form the basis for a legitimate Call-off Contract. A Call-off Contract for one of the suppliers was found on Contracts Finder, it contained no express provision for annual increases. Let’s explore the various guidance and rules on pricing:

Buyers’ Guide on pricing

“You must provide pricing against each individual service offering on the Digital Marketplace.

Pricing principles are set out in clause 3 of the Framework Agreement.

The guidance below, which supports these principles, must be followed when forming your pricing document. This is to ensure that all prices are provided so that a buyer can calculate a final price without needing to contact any supplier:

• If applicable a SFIA rate card may be used as a pricing document.
• Listing a minimum price only is not permitted.
• Pricing documents should clearly lay out complete pricing models and any form of discount structure. This should include applicable unit prices and volume discounts.
• Published prices should be the price that a buyer will pay for the services. They may be reduced, but any reductions must be published on the Platform to be available for all Buyers
• Price ranges are not permitted (unless clearly documented to explain what is included for each price, for example: £100=x £200=x £300=x not to be written as £100 – £300)
• Non-published pricing is not supported, including:
  • Price on Application (POA) is not permitted
  • Prices “from £x per day” are not permitted.
• Maximum prices must be displayed clearly on GCloud 13

Your service offer/offerings and pricing must remain fixed for the duration of the framework and cannot be materially changed or negotiated. You are not permitted to increase prices at any time unless a price review is agreed by both parties prior to the award of the call-off contract.

You are allowed to reduce the service offering price at any time during the life of the agreement, this could be permanently or a time limited offer. Any reduction in pricing must be approved by CCS, updated on the Digital Marketplace and must be available to all customers.

It is not permissible for you to negotiate with customers and/or provide individual price reductions or discounts.”

“You can contact suppliers to ask them to explain their service description, terms and conditions, pricing or service definition documents.

You must not negotiate with suppliers about the details of their service. If it isn’t in their service description, you can’t ask a question about it.”

Framework Agreement Pricing Clauses

3.1 The Supplier agrees that the prices and other terms quoted in its Platform Application will not be increased during the Term of this Framework Agreement, but Suppliers may reduce any of their G-Cloud prices at any time.

3.2 The pricing of Call-Off Contracts must be based on the most up-to-date prices on the Supplier’s Platform Service page.

3.3 Discounted pricing periods may be considered on an individual basis, and subject to CCS approval.

3.4 Subject to clause 3.1, once the G-Cloud Services have been ordered by a Buyer, the Supplier must maintain the Supplier Terms, including the pricing in the Supplier’s Platform entry at the time of the Order, for the length of any Call-Off Contract unless the Call-Off Contract allows for price review.

Clarification statements regarding pricing

Further guidance on pricing is provided in response to ‘clarification questions’, here are some of the statements:

“Listing a minimum price is not permitted. Pricing documents should clearly lay out complete pricing models and any form of discount structure. This should include applicable unit prices and volume discounts. Price ranges are not permitted, however multiple rates can be included on price lists or SFIA rate cards.”

“The SFIA card should not be modified in any way.”

Question: “Whilst I note prices cannot increase during the Framework, is it possible for the published pricing to have annual escalation in it e.g year 1 price is [x], year 2 price is [x+ y%] etc etc?”

Response: “Clauses 3.1 to 3.4 of the Framework Agreement – Attachment 7 set out conditions relating to pricing. In accordance with clause 3.1 of the Framework Agreement, prices quoted in the Platform Application may not be increased during the Term of the Framework Agreement. Framework clause 3.4 allows Call-Off Contract price reviews, if agreed with the Buyer in the Call-Off Contract.”

“No price review limits have been defined, but acceptance of any price review, how it will be calculated, or any limit is at the discretion of the Buyer. Framework Clause 3.1 in Attachment 7 – Framework Agreement relates to the prices published on the Platform, these prices must not increase during the Term of the Framework Agreement.”

“A pricing document for each service listing is mandatory. Any service without a pricing document will be regarded as incomplete and will be removed from the Digital Marketplace. If applicable, a SFIA rate card may supplement a pricing document”.

Table comparing relative compliance with pricing rules

Examples of non-compliance on pricing

• No pricing given at all
• POA for all or significant parts of offering
• Prices given in the form: “from £xxx”
• Prices given in bands (some very large intervals)
• SFIA rates given in bands
• Prices or SFIA rates subject to annual increases
• Pricing is said to be “indicative only”

One relatively common additional failing is where the supplier discloses a full price list of many modules, components and services but has a Service Definition that does not explain the configuration options required to operate the service. So, a cost cannot be calculated.

Service Definition

Suppliers’ Guide states that it ‘may’ include:

• What the service is
• The levels of data backup and restore, and disaster recovery you’ll provide, such as business continuity and disaster recovery plans
• Any onboarding and offboarding support you provide
• Service constraints like maintenance windows or the level of customisation allowed
• Service levels like performance, availability and support hours
• After sales support
• Any technical requirements
• Outage and maintenance management
• Hosting options and locations
• Access to data (upon exit)
• Security
• Link to Website

The Buyers’ Guide includes:

Work with someone who will use the service, buying specialists and technical experts to prepare a list of ‘must-haves’ and ‘wants’.

Your assessment should be based on the information in suppliers’ detailed service descriptions.

You can contact suppliers to ask them to explain their service description, terms and conditions, pricing or service definition documents.

You must not negotiate with suppliers about the details of their service. If it isn’t in their service description, you can’t ask a question about it. (My emphasis)

When choosing the winning service, you should consider:

• Whole life cost – the cost effectiveness, price and running costs of the service
• Technical merit and functional fit – for example, coverage, network capacity and performance
• After-sales service management – the helpdesk, account management function and assurance of supply of a range of services
• Non-functional characteristics – for example, supplier terms, help with onboarding and offboarding, scalability, reliability and automatic disaster recovery

Table comparing relative adequacy of Service Definitions

The Service Definition has a lot of work to do. It must enable the evaluation team to check off must-have and nice-to-have requirements, detail all components from implementation to termination and data extraction and all through-life professional services. Together with the pricing document it should enable whole life cost to be established.

In assessing the adequacy of the Service Definitions I found 37% were so bad that a buyer could not possibly perform this assessment exercise.

Specific faults:

• Simply a duplicate of the Service Summary, Features & Benefits
• Thin advertising brochure or sales powerpoint
• Document completely unrelated to the service
• None or few of the items listed in the Suppliers’ Guide
• Detailed list of features, then a statement that the service may not have them all
• Sloppy, unprofessional, incoherent, many typos, malapropisms, bad English

Accessibility

Accessibility is of increasing importance:

“WCAG 2.2 AA is the new minimum accessibility standard for all UK Government public sector websites and mobile apps.” [Link]

While it is not clear from the above article if that applies only to citizen access to public sector applications, in my experience many Central Government departments also expect it for their staff’s applications.

Of the Control Group of ten successful SMEs, three, with combined sales of £4.1m on G-Cloud 13, responded to the Accessibility question in “Using the Service” as “None, or don’t know”.

Of the population of unsuccessful SMEs sampled 70% responded to the Accessibility question in “Using the Service” as “None, or don’t know”. This cannot be classed as a ‘fatal error’ given the sales in the Control Group. While I consider it a very serious failing, certainly for Central Government, I do not know if it is perceived as such for (say) NHS, Local Authorities or Blue Light.

So, I have marked services expressing “none/don’t know”, for this part of the exercise as 40/100 and WCAG 2.1A as 80/100 (i.e. capable of remediation for G-Cloud 14) and WCAG 2.1AA+ as 100/100.

Table marking approach to Accessibility

Standards (e.g. ISO27001, Cyber Essentials, C STAR)

Cyber security standards (see Appendix 2 for a brief outline of the different standards discussed here) are going to be an increasingly important attribute. For G-Cloud 14 be aware that CCS are reported to not consider ISO 27001 a substitute for Cyber Essentials +, but that a supplier wanting to win contracts on Lot 2 should have both.

Table marking approach to Cyber Security

The weak performance of the main sample does indicate that this could be a strong indicator of cause. This is likely to be a significant impediment to success on G-Cloud 14.

One additional matter to note. A supplier of Cloud Software (not a reseller) requires their own accreditation. That the provider of hosting services bundled with the service has accreditation, which it should have, is not sufficient to cover the software developer.

Terms & Conditions

Terms and conditions are, almost universally, badly presented. Typically, they will be a form of software or Cloud software contract used in the private sector and contain many conflicts with the Framework Agreement, Order Form, Call-off Contract and the suppliers catalogue entry on G-Cloud and attached documents.

Some will include strongly worded clauses purporting to state that they represent the sole and entire details governing the relationship.

The Framework Agreement states:

8.3 If there is any conflict or ambiguity between the clauses of this agreement, to the extent necessary, the order of precedence for resolving the conflict is:

8.3.1 the Framework Agreement

8.3.2 the completed Order Form

8.3.4 the clauses of a Call-Off Contract (excluding Supplier Terms)

8.3.5 the Supplier’s Terms

8.3.5 any other document referred to in the Call-Off Contract clauses

It remains to be seen whether CCS will attempt to reduce the opportunity for misunderstanding in G-Cloud 14. I am not qualified to advise on the law and refrain from doing so, but on marketing grounds I would argue that using a full commercial contract with conflicts is a mistake and may be limiting opportunities.

Overall Quality & Standard of Presentation

There are some very well-presented services in the sample tested and there are many that are adequately fit-for-purpose, these just need one or more of the impediments noted in the preceding pages to be removed and G-Cloud Buyability™ will have been restored. But there are more examples of poor presentation than I was expecting. This will impact the selection and assessment process no matter how much a buyer is encouraged to avoid bias, because it is potentially a signal of a culture of a lack of care, professionalism and understanding of the procurement process.

It is also, I suspect, a root-cause of some of the errors, fatal or limiting which I have noted. Here is a list of examples:

• Misspelling, malapropisms, bad English throughout
• Empty or completely wrong documents uploaded
• Service Definitions 1 or 2 slides with very little content
• Paragraphs unfinished, some just a heading and no text
• Framework agreement & guidance wholly ignored
• Minimum term x years (5 the longest), autorenewal x years
• Social Value: “Not Applicable” or “Contact Us for Detail”

UK Tech SMEs are and will be the foundation of the UK’s sustainable competitive advantage on which growth, jobs and future prosperity depend. How is Social Value ‘not applicable’?

Conclusion

I last performed a review of similar nature to this in 2016, many of the conclusions and discoveries then are still with us today. This is disagreeable as it reflects a huge waste of time and expense by the UK tech SMEs we most want to succeed in this highly competitive marketplace.

It has a deeper cost. It is denying the public sector full access to the innovation, services and financial benefits working with our SMEs would provide.

£6 billion has been spent with SMEs on G-Cloud and £1.3 billion in the 12 months to September 2023. The successful cohort of SMEs, micro, small and medium, show what can be done. Except for a tiny number of SMEs who are attempting to sell out of scope products and services, the errors and misunderstandings which are catastrophically preventing success can be resolved. These problems are best resolved now, at the commencement of G-Cloud 14 when it is incumbent on us all to take a fresh look at the rules and guidance and translate this into how we address the market.

Detail and a detailed understanding is paramount. Your goal is to be selected and the last supplier standing after the elimination round.

If you do not have the time or enthusiasm to research and design your application to achieve this goal – then take advice!

Appendix 2 – Summary of Cyber Security Accreditations

ISO 27001 Main Features

• International standard for information security management systems (ISMS).
• Provides a comprehensive approach to security, covering people, processes, and technology.
• Requires a systematic examination of an organization’s information security risks, including threat, vulnerability, and impact assessments.
• Implements a coherent and comprehensive suite of information security controls tailored to the organization’s needs.
• Requires continuous monitoring, review, maintenance, and improvement of the ISMS.

Overlaps: Emphasizes a risk management process, which is a concept also found in CSA STAR and, to some extent, in the principles underlying Cyber Essentials.

Differences: More comprehensive and flexible than Cyber Essentials and Cyber Essentials Plus, requiring a formal risk assessment and the implementation of a tailored set of controls. It’s also internationally recognized, whereas Cyber Essentials is specific to the UK.

Cyber Essentials Main Features

• UK government-backed scheme designed to help organizations protect against common cyber-attacks.
• Focuses on five basic security controls: secure configuration, boundary firewalls and internet gateways, access control and administrative privilege management, patch management, and malware protection.

Overlaps: Shares a foundational approach to cybersecurity with ISO 27001’s emphasis on risk management, but at a more basic, accessible level.

Differences: Less comprehensive than ISO 27001. It is specifically aimed at providing a basic level of protection and does not require the same depth of documentation or management processes.

Cyber Essentials Plus Main Features

• Builds on the Cyber Essentials scheme by adding an additional layer of assurance through the independent testing of the organization’s security controls.

Overlaps: Includes all the requirements of Cyber Essentials, with the added rigor of external testing.

Differences: While still more basic than ISO 27001, the requirement for independent verification introduces an element of third-party assessment similar to the certification process for ISO 27001. However, it remains focused on a limited set of controls.

CSA Security, Trust & Assurance Registry (STAR) Main Features

• A comprehensive program for cloud service providers to demonstrate security controls.
• Incorporates key principles from ISO 27001 but is tailored specifically for cloud services.
• Offers a three-level program (self-assessment, third-party audit, and continuous monitoring) to certify cloud security capabilities.

Overlaps: Shares a broad approach to risk management and security controls with ISO 27001, especially at the higher levels of the STAR certification, which include third-party audits.

Differences: Specifically designed for cloud services, making it more relevant for SaaS providers than the more general ISO 27001. The tiered certification process also offers a path from self-assessment to continuous monitoring, which is unique among these frameworks.

Summary

ISO 27001 is the most comprehensive, suitable for organizations looking for a robust ISMS that covers all aspects of their information security.

Cyber Essentials and Cyber Essentials Plus provide a straightforward, focused framework for UK businesses to protect against common cyber threats, with the Plus version adding a layer of verification.

CSA STAR is specifically tailored for cloud service providers, offering a way to demonstrate security practices in a cloud environment, with a focus on transparency and trust in cloud computing.