Striking Balance between Public Procurement and Innovation

As digital solutions become increasingly integrated into government services and benefit delivery, public administrators can use the public procurement process to better leverage technology to improve systems, enhance efficiency, and deliver better outcomes for residents. The public procurement process for technology solutions is complex, requiring careful planning, evaluation, and decision-making. Prioritizing responsible, equitable design and regulation of technology through a cross-sector approach is a core tenet of public interest technology, ensuring that digital solutions are developed and implemented in alignment with public values and needs because more public services depend on digital solutions. New America’s Digital Impact and Governance Initiative and Public Interest Technology University Network partnered to convene a roundtable discussion in March 2023 to gather insights from academic experts on the public procurement of digital systems. The framework for the discussion was focused on the opportunities and challenges associated with the administration of public services or benefits on the state level. This work builds on the Reconceptualizing Public Procurement to Strengthen State Benefits Delivery and Improve Outcomes research New America published in October 2022.

This synopsis of findings from the roundtable cites six key challenges to public procurement of digital systems for benefits delivery and civil service in states and provides six corresponding recommendations to improve public procurement processes. By understanding and addressing these barriers, state governments can better leverage technology to improve public benefits delivery and enhance the well-being of their residents.

State governments across the country have started implementing emerging data technology, including artificial intelligence and automated decision systems, to try to enhance the administration of public service delivery with digital solutions. Many governments are motivated to harness innovation and modernization in ways to better serve the public despite budget challenges, limited capacity, and complex responsibilities. It is important to note as well that government innovation takes place in an environment where failed digital solutions can be perceived as a misuse of public funds or a deficiency in leadership. Often, the risks necessary for people-centric solution approaches and outcomes in innovation are limited by a government’s need to mitigate harm and maintain operations.

Acknowledging these challenges, many governments are exploring ways to balance the risks and rewards of digital transformation efforts. This includes partnering with private sector entities, collaborating with other government agencies, and engaging with the community to ensure that innovation aligns with public values and needs. Through innovation and investment in digital public infrastructure, governments can unlock the potential of digital solutions and emerging technologies while maintaining trust and confidence in government services.

The field of public procurement has undergone rapid change in recent years, with digital innovation playing a key role in transforming the procurement process. From e-procurement systems to user-friendly digital processes, technology has the potential to revolutionize how governments purchase goods and services. However, despite the promise of these innovations, many public procurement agencies face significant barriers to adopting and implementing them. By understanding these barriers, we can identify strategies to create more efficient, effective, and transparent procurement processes.

1. Restrictive Risk Management in Public Service

Public sector risk management and procurement practices often prioritize cybersecurity, compliance, and predictability of project milestones over testing and experimentation. This approach makes it difficult to implement digital technologies that can better respond to constituents’ needs. Additionally, some procurement requirements such as nondisclosure agreements and proprietary solutions can further complicate the process of bringing new approaches into government. In order to effectively manage risk in the procurement process, governments must carefully evaluate the potential risks and benefits of new technologies before adopting them. This involves conducting thorough security assessments and ensuring that any new technology solutions align with government standards and regulations while keeping an eye toward the benefits of a new technology or approach, even though such due diligence can slow down new technology adoption.

Another significant risk associated with adopting new technology in government IT departments is the challenge of managing it using existing personnel and their current tech management tools and staying within operational budgets. IT departments tend to prefer familiar technologies as that helps maintain a manageable level of required in-house competencies. However, this approach disincentivizes the adoption of new technology.

Risk management in government procurement has also suffered from insufficient resources and support and is often approached from a tactical perspective, focusing on the immediate, transactional elements of the process rather than considering it as a strategic lever to achieve the overall mission and vision of an agency or department.

2. Asymmetric Relationships between Governments and Vendors

A better understanding of technical objectives that digital solutions aim to achieve, and how much money and time it should reasonably take to achieve them, is needed in order to create a level playing field with the private sector, which tends to overstate the capabilities of technology and downplay its drawbacks. When vendors perceive governments as a captive market, where governments have limited or no alternatives to a particular product or service, there is less interest or motivation to provide thoughtful and nuanced solutions. Often procurement technology solutions may inadvertently consume more time than expected, creating a confusing experience for both buyers and vendors.

Governments, viewing the ineffectual solution as sunk cost, then struggle to hold business consultancies and service providers accountable for failing to meet their promised deliverables. Vendors in turn lack genuine financial incentives to enhance their products or services, which leads to a continuing cycle of subpar services.

Moreover, vendors that offer low-cost, off-the-shelf solutions can generate unanticipated long-term operational costs and create path dependencies. This situation effectively locks state governments into extended relationships with vendors that have not met their needs. This also creates further lock-ins, such as data and code ownership, where governments end up not owning the data for the services they procured. Such dependencies hinder the ability of governments to adapt and upgrade their infrastructure, and ultimately have a negative impact on the quality and efficiency of public services.

3. Disconnect between the Federal Government and State Governments

State governments respond to citizens’ needs and provide public services in myriad ways, but when the federal government allocates funds for programs without offering sufficient guidance, states face challenges in adapting their procurement processes accordingly. The federal government, for example, often falls short in assisting state and local governments with the administration of social benefit programs by primarily providing funding and regulations but offering limited guidance for implementation. When procurement changes occur at the state level, cities are left to adapt and comply with these new policies on their own, with little to no human capital. These federal mandates exert a significant influence on future procurement decisions at the state level by shaping the direction and priorities of state and local governments. This top-down approach can lead to solutions that are not tailored for individual communities.

Additionally, while local government solutions can serve as effective models, they may not always be easily scalable to larger jurisdictions. Locally based solutions might be specifically designed to address the unique challenges of a particular community, and their success may not translate seamlessly when applied in a different context. Essentially, procurement decisions cannot be one-size-fits-all; the federal government must provide guidance while offering some flexibility for local governments to adopt the best practices for their constituents.

4. Outdated Bureaucracy

Current bureaucracies overseeing procurement often rely on selection criteria that were established under different circumstances and have not kept pace with the rapid growth and evolution of technology. Consequently, these prescriptive criteria are being used to evaluate cutting-edge technologies, leading to suboptimal outcomes and stifling innovation. Decisions are often influenced by factors such as prior sales to other government agencies and familiarity with a vendor or a vendor’s ability to navigate often archaic bureaucracies, which can lead to an unsuccessful outcome of the procurement process.

Furthermore, the emergence of new technologies creates challenges in training and capacity. Too often, the solutions are outpacing a government’s ability to harness the latest advancements, limiting the government’s ability to assess new proposals effectively. This in turn leads to dysfunction where technical individuals educate an ever-growing list of nonexperts about technology. The imbalance makes it more challenging to access independent experts in the field and can cause significant delays, inefficiencies, and suboptimal outcomes in the procurement process for government entities and their constituents.

5. Lack of Competitiveness and Accessibility for Smaller Firms

Large, incumbent vendors tend to dominate the field of public procurement, often sidelining smaller, more innovative firms from the process. As a result, small vendors at the forefront of digital technologies may be reluctant to pursue collaborations with a government entity, leading to missed opportunities for the public sector to leverage cutting-edge solutions.

The competitive landscape, coupled with bureaucratic requirements, can make it challenging for smaller tech firms to participate in government procurement processes. The complexity and lengthy nature of these processes can discourage or hinder firms from competing effectively against established industry players. These smaller vendors, which are more likely than established vendors to be minority-, women-, queer-, and veteran-owned businesses, may also be reluctant to put forth highly innovative proposals, fearing that deviating significantly from the specified requirements could jeopardize their chances of securing the contract. In addition, prescriptive procurement often stifles innovation, and providing insufficient information and resources that clearly define the procurement process can create difficulty for new vendors.

Due to the time and financial resources required to complete the procurement process, smaller companies with less budget and human capital than big, established competitors cannot afford to participate. Grant funding, external investors, or other forms of backing can mitigate these concerns, but they aren’t enough. Balancing the relationship among pricing structure, access (in terms of cost and open access), contract negotiations, and investments presents a significant challenge. This is particularly crucial for smaller firms seeking to penetrate the government market, where competitive pricing can be a determining factor in procurement decisions.

6. Limited Technical Competencies in Public Offices

There is limited technological literacy among some of the public offices responsible for making critical decisions in public service delivery and procuring digital systems to enable them. The complexity of the systems being purchased and their long-term impact requires a higher level of technical competencies to ensure informed decision-making. This is not simply a matter of choosing between one product or another, but rather evaluating different smart systems and approaches with an eye toward both short- and long-term gains and losses.

Vetting responsibilities are sometimes delegated to IT departments, which often do not understand procurement IT software or lack the necessary skills to implement and maintain new systems. Even when an IT department is responsible for the operation of these systems, its teams are often not a part of the decision-making process at the procurement stage.

Recruiting and retaining specialized labor in the public sector continue to be a challenge where digital skills have not traditionally been part of professional development within the government sector. This issue is not unique to procurement offices but is also reflected in the types of questions posed by state and federal procurement officers, who lack the necessary knowledge to make informed decisions. Often, these decision-makers have no timely access to up-to-date technical education and training—or have no access to such education and training at all—to help them vet and understand the intricacies of emerging technologies and their potential impact.

In light of the increasing importance of technology in public benefits delivery, this section offers recommendations and insights to improve the public procurement of digital systems for benefits delivery in states.

1. Involve the Public in the Procurement Process

Incorporating public input in the procurement and digital solution development process is vital for ensuring that public needs and preferences are adequately addressed. By involving citizens in public procurement, governments can foster a sense of ownership, shared responsibility, transparency, and accountability. This participatory approach should extend to industry and the wider community as well, fostering a collaborative environment that values diverse perspectives and allows for better alignment with the needs and values of the public. Involving the public in the public procurement process not only increases transparency and accountability but also decreases the risk of procurement decisions that do not align with the needs and values of the communities they are directly impacting.

Procurement offices can leverage public input through human-centered design techniques and user research methodologies to foster innovation and digital solutions designed by, and tailored to, the needs of the very people they are intended to serve. By investing in and encouraging solutions developed with end users, governments can benefit from a more user-centric approach that aligns with the real-world requirements of their constituents.

In addition to supporting externally developed innovations, governments must invest in their own internal innovation capabilities by developing and nurturing the technical capacities of staff. By cultivating a culture of continuous improvement and experimentation, public sector organizations can stay at the forefront of emerging trends and technologies, effectively adapting to the evolving needs of communities.

2. Foster Direct Collaboration between and within Government and Vendors

Interdisciplinary knowledge-sharing and collaboration play a crucial role in enhancing outcomes in public procurement. A more comprehensive understanding of requirements and potential solutions can be achieved by involving all stakeholders, including government officials, vendors, and end users, in the procurement process. Stakeholders can also include non government and nonvendor experts from academia, civil society, and community organizations. Simple activities such as joint sessions, open office hours, and workshops provide valuable opportunities for vendors to demonstrate and introduce their offerings. Websites, FAQs, templates of forms, and guides for how to complete forms can also increase vendor supply.

Supporting new avenues for innovative vendors to participate in public procurement including design competitions, agile contracting, and open consultation processes can significantly increase the supply of digital services. Seeking community-led collaboration can also expand the accountability and transparency to distribute power dynamics and foster an open dialogue among all parties, helping to ensure equitable access for smaller firms, which are also more likely than large, established companies to be minority-, women-, and veteran-owned.

Emphasizing interdisciplinary conversations and cross-team collaboration is essential to ensure that procured products align with their intended purpose. Involving legal experts in the process is essential, as they must work closely with procurement teams to guarantee that the purpose of the product aligns with the final purchase decision.

Creating shared pools of expertise and management resources can also help hold vendors accountable while fostering a collaborative environment. Establishing mechanisms that enable different jurisdictions to work together can be a powerful tool in enhancing the overall effectiveness of public procurement. Overcoming the challenges of setting up such collaborative frameworks is vital for governments so they may harness the full potential of cross-jurisdictional cooperation and ensure that procurement decisions produce the best possible outcomes for their communities.

3. Improve Support from the Federal Government to State and Local Governments

The federal government has the potential to offer enhanced support to state and local governments in various ways that help transition from policy to implementation while preserving state and local autonomy. By promoting shared digital solutions and components, whether developed internally or procured externally, the federal government can help create a cohesive and unified approach to addressing common technology needs. Instead of state and local governments separately tackling their often similar technological challenges, federal support can foster collaboration and resource-sharing among these entities.

Additionally, the federal government can provide guidance on data formats and standards to encourage interoperability between systems and promote seamless information exchange. This can ultimately lead to improved efficiency and more effective collaboration among different levels of government. By conducting baseline design and research, the federal government can provide valuable insights that inform state and local solutions, ensuring that they are well aligned with overarching national strategies and objectives.

The federal government can also facilitate collaboration between federal lawyers and their state and local counterparts to clarify interpretations of laws and regulations, thus promoting a consistent understanding of legal requirements across jurisdictions. This holistic approach to supporting state and local governments can significantly enhance the effectiveness and resilience of public sector digital infrastructure.

4. Prioritize Accountability and Flexibility

While contract law traditionally emphasizes autonomy and adherence to the terms agreed upon during the signing process, procurement contracts in the public sector must allow for enhancements to happen in an iterative manner, similar to how private sector tech benefits from flexibility. This adaptability fosters collaboration between parties, rather than promoting a rigid, transactional give-and-take relationship that might hinder the effective implementation of digital solutions. New procurement models such as minimum viable processes and agile procurement can significantly improve current practices.

Accountability in public procurement should extend to both vendors and government decision-makers. Vendors, which occasionally exploit the procurement process, must be held responsible if they fail to deliver on their promises and provide poor quality products and services. Simultaneously, government decision-makers must be held accountable by learning from past mistakes and making informed choices that best serve the public interest. By ensuring that all parties involved in the procurement process are held accountable, governments can create a transparent and equitable procurement environment that encourages collaboration, fosters innovation, and ultimately leads to better outcomes for citizens.

5. Value Results over Cost Savings

Procurers require greater financial flexibility to evaluate and select bids based on their overall suitability, rather than focusing solely on the lowest price. Leveraging open source and open technology solutions can also expand the decision-making criteria and support more informed choices that take into account a broader range of technical factors.

There needs to be a shift in focus away from “low-cost purchasing” and toward outcome-oriented purchasing, which addresses the needs of the wider stakeholder community, rather than merely prioritizing low-cost options that address immediate concerns. This includes revising government evaluative criteria such that cost savings is measured not only by the bottom line, but also through other factors, such as year-over-year sustainability; the flexibility to move to a new vendor after the first contract is complete; and the annual cost of maintenance, training, and support. This approach can lead to sustainable and effective procurement decisions that benefit both governments and the communities they serve, while also opening the door to small vendors in the forefront of digital development.

To further enhance the procurement process, conducting thorough research on vendors is essential. By examining the quality of their previous work and relationships with clients, governments can gain valuable insights into the vendors’ capabilities and track record. These vendor ratings can increase competition and encourage procurement professionals to share experiences that can help break the cycle of unproductive actions and partnerships.

6. Develop Stronger Technical Capacity within the Public Sector

Capacity building in the public procurement sector requires better professional development opportunities, as well as the establishment of tailored technical positions in procurement offices. It is also vital to increase the diversity of expertise within procurement teams by fostering the development of education programs specifically designed for roles that intersect policy, technology, and end users in the procurement process with a public interest technology frame. Such an educational framework would help to address diversity, equity, inclusion, and accessibility practices both in the offices leading the procurement efforts, and the solutions they procure. This educational framework needs to train existing personnel on technical skills, create job titles that attract new technical talent, provide new educational pathways that include a public interest technology framework, and update civil service exams to include digital literacy knowledge, skills, and abilities (KSAs).

By fostering the development of tailored education programs on technology fundamentals for procurement professionals, which involves analysis of end users’ needs, governments can better ensure that digital solutions are designed with a comprehensive understanding of the real-world challenges faced by the people they serve. This, in turn, leads to more efficient and satisfactory public services that cater to the diverse needs of the community.

Establishing a stronger connection between the systems developed and the people who use them is also essential for achieving optimal outcomes. By bridging this gap, governments can shift the balance of power toward citizens, making user feedback an invaluable resource in understanding the barriers faced in accessing government services without sacrificing privacy.

Public administrators face a critical balancing act between using their available resources to provide essential services to residents and pushing for public sector innovation to drive progress and meet changing needs. While it is crucial for governments to prioritize the efficient delivery of services, avoiding experimentation and iterative processes can result in stagnant systems that fail to keep up with changing needs. At the same time, prioritizing a new approach at the expense of service delivery can create inefficiencies and gaps in essential services.

Government officials may need to prioritize improving their current public procurement process over pursuing the “bleeding edge” of technology as it may introduce unnecessary risks and complexities and is not always the most suitable approach for public sector projects. However, by prioritizing improvements in the digital solution procurement process such as greater collaboration between the public, vendors, and government jurisdictions, public administrators can deliver more effective, reliable, and adaptable public services.

To navigate this balance, governments must take a more comprehensive and risk-informed approach throughout the procurement process to ensure that digital services efficiently and effectively meet the public’s needs. Striking the right balance between thoughtful exploration of better solutions and building upon best practices and more agile solutions, governments can drive progress, meet the needs of the public, and promote prosperous and thriving societies.

This work is supported by the Bill & Melinda Gates Foundation. The team is especially grateful for the support and partnership of Ryan Rippel, Caira Woods, and Yuri Kim, who are leading the Gates Foundation’s U.S. Economic Mobility and Opportunity (EM&O) portfolio with a goal of moving more people out of poverty over the next decade.

The authors appreciate the passion and perspectives of the entire Digital Impact and Governance Initiative (DIGI) and Public Interest Technology teams. Thank you to the following individuals who helped along the way and who we hope will continue making contributions to the emerging field of digital public infrastructure: Alberto Rodríguez Álvarez, Vontisha Fludd, Sarah Forland, Allison Price, Summer Boucher-Robinson, and Andreen Soley.