Skip to content

The Heilmeier Catechism

The Heilmeier Catechism is a set of nine questions that was developed by George H. Heilmeier, a former director of the Defense Advanced Research Projects Agency (DARPA), to evaluate the value and feasibility of a proposed research project or technology development effort. The questions are designed to clarify the objectives, risks, and potential impacts of the proposal.

Let’s break down each question:

  1. What are you trying to do? Articulate your objectives using absolutely no jargon.
    This is asking for a clear, concise statement of the project’s goal or objective, stripped of any technical jargon. It’s important that anyone, regardless of their technical background, can understand what the project is aiming to achieve.

  2. How is it done today, and what are the limits of current practice?
    This question seeks to understand the existing state of affairs. It asks for an explanation of current methods, technologies or practices that address the same problem or need, and their limitations. This sets the stage for understanding why a new approach or solution is necessary.

  3. What is new in your approach and why do you think it will be successful?
    This question asks for the unique elements or innovations in the proposed project. It’s looking for what differentiates this project from existing solutions, and why those differences will lead to success.

  4. Who cares?
    Here, the question is about the project’s stakeholders. Who will benefit from this project? Who are the potential users or customers? Understanding who cares about the outcome of a project helps justify the effort and resources that will be needed to carry it out.

  5. If you are successful, what difference will it make?
    This question seeks to understand the potential impact of the project. What will change if the project is successful? This could include technical impacts, as well as broader impacts on society, economics, or other fields.

  6. What are the risks?
    This question asks for a candid assessment of the risks involved in the project. This includes technical risks, such as the potential for failure or unexpected obstacles, as well as other types of risks, such as financial, ethical, or societal.

  7. How much will it cost?
    This question is about the project’s budget. It’s asking for an estimate of the financial resources that will be required to carry out the project. This includes direct costs, like materials and labor, as well as indirect costs, like overhead or administrative expenses.

  8. How long will it take?
    This question is asking for a timeline. How long will it take to complete the project, or to achieve the key milestones? This helps assess the feasibility of the project and plan for the necessary resources.

  9. What are the mid-term and final “exams” to check for success?
    This question asks for the project’s evaluation criteria. How will you know if the project is on track, or if it has been successful in the end? These could be technical metrics, user feedback, market adoption, or any other measures that indicate the project’s progress and success.

These questions help researchers, innovators, and funders to think critically about the purpose, value, and feasibility of their projects. They can be a useful tool for planning, evaluating, and communicating about technology development efforts.

Example: applying it to the Nua project

  1. What are you trying to do? Articulate your objectives using absolutely no jargon.

Nua is aiming to create an open-source system that helps organizations manage their web applications from start to finish. This includes everything from the development and packaging of the applications, to their deployment, maintenance, and security.

  1. How is it done today, and what are the limits of current practice?

Currently, organizations use a variety of different tools and systems to manage their web applications, which can be inefficient and expensive. These tools often do not communicate well with each other, leading to duplicated effort and potential errors. The maintenance and updating of these systems can also be a significant burden.

  1. What is new in your approach and why do you think it will be successful?

Nua’s approach simplifies this process by providing a single, integrated system that covers the entire life cycle of a web application. It focuses on using established standards and technologies, making it easier for developers to use. Its design principles aim to reduce unnecessary complexity, improve security, and allow for future expansion. By meeting the specific needs of organizations and developers, and by leveraging the benefits of open-source software, Nua has the potential to be a valuable tool in the market.

  1. Who cares?
  • Small and medium-sized businesses, associations, public services, and any other organizations that use web applications could benefit from Nua. These organizations often lack the resources to manage their web applications effectively and could greatly benefit from a streamlined, integrated solution. This is particularly true for organizations looking to gain digital autonomy by prioritizing open-source software.
  • Developers and software developing organizations (SDOs). Nua will allow them to focus on their core business, rather than spending time and effort on managing their web applications.
  1. If you are successful, what difference will it make?

If successful, Nua will significantly reduce the time, effort, and cost associated with managing web (i.e. modern client/server) applications. It will improve the quality and security of these applications and could enable more organizations to adopt and benefit from open-source software.

  1. What are the risks?

Risks include:

  • The challenge of gaining adoption in a market that is already filled with established competitors.
  • Not being able to keep up with the rapid pace of technological change.
  • Failing to meet the diverse needs of different organizations and developers.
  1. How much will it cost?

The initial budget for the project is ~€500,000, funded in part by Abilian and its customers, in part by a research grant from the French government, and in part by tax breaks. This funding will cover the development of the platform and its initial deployment within Abilian. Additional funding will be required, after the R&D bootstrap phase, for further development and deployment, as well as for marketing and promotion. The total cost of the project is difficult to estimate at this time, but could potentially reach €1,000,000 or more by 2028.

  1. How long will it take?

The project has already been under development for two years. The timeline for full maturity and widespread adoption is uncertain, but could potentially take several more years, given the scope and complexity of the project. An initial “V1” (MVP) release is planned for 2023.

  1. What are the mid-term and final “exams” to check for success?

Mid-term checks include successful deployment and use of Nua within Abilian and a handful of other organizations. Feedback from these early adopters will be crucial.
The final “exam” will be widespread adoption and use of Nua by a wide range of organizations, with demonstrable benefits in terms of efficiency, security, and digital autonomy.

Additional questions

Additionally, specific question can always be added based on the nature of the project and the context in which it’s being developed. Here are some examples:

  1. Why is addressing the problem you’ve identified crucial and currently unfulfilled in our society? This question highlights the importance of the problem the project aims to solve. It asks for an understanding of the societal context and the existing gap that the project proposes to fill. This is a critical aspect because it provides justification for the project’s existence, its relevance, and its potential impact.

  2. How does your project align with broader trends or developments?
    This can help to understand the future relevance and adaptability of the project. It can also highlight potential synergies or conflicts with other ongoing efforts or trends.

  3. What alternatives have been considered?
    This question prompts a discussion about other solutions that were explored before deciding on the current approach. This helps ensure that the chosen approach is indeed the best one and not just the first one that came to mind.

  4. What is the scalability of your solution?
    Understanding how a solution can scale is crucial, especially in the context of technology and software. Can it handle more users, more data, or more complex tasks as it grows?

  5. What ethical considerations are associated with your project?
    This is particularly important in today’s world, where technology can have significant societal and ethical impacts. It’s crucial to consider these aspects from the outset of a project.

  6. What is your plan for disseminating or implementing the results of your project?
    Research and development don’t exist in a vacuum. It’s crucial to think about how the results will be shared, applied, or commercialized to achieve their intended impact.

  7. How will your project be sustained in the long term?
    This question can uncover plans for maintaining and supporting the project after its initial development phase, which is particularly relevant for software projects or any project where ongoing support or updates might be needed.

  8. Why are you and your team the ideal candidates to execute this project? This question emphasizes the importance of the team’s skills, experience, and dedication. Not only does it focus on the technical abilities needed to execute the project, but it also delves into the softer aspects such as the team’s passion, resilience, and commitment. This is crucial because the success of any project or startup is often heavily dependent on the capability and drive of the team behind it.

  9. How will you obtain the necessary resources (financial, human, etc.) to complete the project?
    This question pushes for a practical plan to gather the required resources. Many projects fail due to lack of resources rather than lack of technical feasibility.

  10. What is the value proposition of your venture? This question directs attention to the benefits that the project will deliver. It seeks to understand what value will be created, how it compares to existing solutions, and why users or customers would choose this new solution. This is vital in understanding the potential marketability and scalability of the project or venture.

Comment

If you can’t explain it simply, you don’t understand it well enough.
(Albert Einstein)

Heilmeier’s Catechism is just as potent a tool for investors assessing groundbreaking ventures or public servants assessing R&D funding proposals as it is for entrepreneurs who are in the process of creating and building these ventures.

These questions may seem basic and universal at first glance. Yet, providing answers necessitates innovators to have a profound comprehension of their technology and project.

Here’s a piece of actionable advice: attempt to answer these questions for your project/initiative/startup. Then, explain your answers to your friends, family members, trusted colleagues or investors, regardless of whether they have a technical background or not. You may be taken aback by the difficulty of this task, the insights you gather from their feedback, and how much more adept you become at understanding and articulating your venture.

Criticism

The Heilmeier Catechism is a highly useful tool for evaluating and shaping research or development projects, but like any tool, it is not without its potential drawbacks. Here are a few criticisms that could be made:

  1. Overemphasis on Practicality: The Heilmeier Catechism is highly focused on the practical aspects of a project such as its feasibility, impact, and cost. While these are important considerations, this approach may overlook or undervalue projects that have more theoretical or exploratory goals. This could potentially stifle more innovative or risky research that doesn’t have a clear immediate application but could lead to important breakthroughs in the future.

  2. Simplicity and Brevity: While the simplicity of the Heilmeier Catechism is part of its appeal, it can also be a limitation. The questions are broad and open-ended, and important nuances may be missed in the effort to provide concise answers. Also, some projects may not fit neatly into the framework of these nine questions.

  3. Assumption of Understanding: The Catechism assumes that the researchers have a clear and deep understanding of their project and its context. However, in early stages of research or for complex multidisciplinary projects, this might not be the case. The Catechism does not offer guidance on how to develop this understanding.

  4. Lack of Emphasis on Ethical Considerations: The Heilmeier Catechism doesn’t explicitly ask about potential ethical issues, societal implications, or unintended consequences of the project. As technology becomes increasingly intertwined with society, these considerations are becoming more important.

  5. Rigid Structure: While the Catechism’s structure is helpful for organizing thoughts and presenting them clearly, it may not fit all types of projects or all stages of development. It may not accommodate the iterative nature of some research or development processes.

In spite of these potential criticisms, the Heilmeier Catechism remains a valuable tool when used thoughtfully and in conjunction with other evaluation and planning methods. It provides a strong foundation for thinking about, planning, and communicating about a project.

References

See also

#project #initiative

Page last modified: 2024-11-13 14:01:29