V.B.1. Internal Technical Evaluation: Pros/cons, Go/No-Go Reported

Estimated Execution Time

3 weeks

Objective

To complement the internal evaluation, both technical and clinical, of the innovation solutions gone to the procurement process and at the final stage of its implementation (described in Stage IV). This assessment must go beyond a descriptive report and incorporate an overall progress across the entire innovation pathway, and the credibility and maturity of the provider’s innovation roadmap. This will inform a PROS/CONS analysis and a GO/NO-GO decision from each of the healthcare providers independently.

The objective is to determine the solution’s real-world feasibility, usability, technical robustness, clinical relevance, and alignment with particularities of the healthcare providers, including workflows and priorities.

Who is Involved

Clinical experts and end-users – physicians, nurses, technicians, or other health professionals who used or supervised the use of the solution during the pilot. Evaluate clinical relevance, usability and workflow incorporation.
Technical teams (IT, biomedical engineering, digital transformation units) – responsible for infrastructure compatibility, integration, cybersecurity, and reliability assessments. Assess system performance, integration feasibility and operational sustainability.
Pilot site coordinators – oversee the implementation process, collect data and ensure communication between clinical and technical teams. Provide consolidated data, monitor implementation progress, and ensure methodological coherence.
Procurement and management representatives – ensure alignment with strategic and operational priorities and contribute to the GO/NO-GO decision.

Activities / Tasks

V.B.1.1. Technical, clinical and operational evaluation

Assess system performance during the implementation and pilot phases.
Evaluate clinical usability, workflow impact, user acceptance, and clinical effectiveness indicators.
Technically depending on the innovation solution provided assessment the evaluation could include: stability, reliability and downtime, integration with existing systems (HER, data platforms, devices, etc.), cybersecurity and compliance (GDPR, data governance), scalability and interoperability, maintenance and technical support required.
Clinically depending on the innovation solution provided assessment the evaluation could include: improvement in quality of care or patient outcomes, impact on staff workload and workflow efficiency, safety considerations, ease of use and training requirements, fit with clinical guidelines and protocols.
Compare technical performance with initial specifications and tender expectations.
Identify technical risks, unresolved issues, and long-term operational implications.
Identify clinical limitations, operational challenges, or barriers to adoption.

V.B.1.2. Evaluation of the complete the innovation pathway

This evaluation focuses not only on the end product but on the entire innovation journey, including co-creation, adaptability, and learning capacity throughout the procurement project.
The evaluation assesses the solution’s progress from early concept to the pilot maturity reached and should determine:
- Whether the innovation has achieved the maturity level expected at this project stage (e.g., IML or TRL advancement).
- How well the development aligned with the initial innovation objectives and identified needs.
- The level at which the provider addressed key milestones defined in the initial tender or technical specifications.
- Evidence of validation in real operational contexts.
- The robustness of iterative development cycles (e.g., responsiveness to feedback, speed of iteration).
Evaluation of proposed roadmap of the innovation provider, that includes different dimensions, strategic, regulatory and others:
- Innovation provider capacity – stability, business model sustainability and long-term sustainability.
- Regulatory and compliance pathway – progress toward required certifications (e.g. CE MDR, CE IVD), documentation maturity and regulation risks, including reimbursement strategies and barriers.
- Competitive potential and market positioning – Whether the innovation solution proposed still has a competitive advantage and the risk of obsolescence (that will be informed by the market report (V.C.1)
- Strategic alignment – Fit with the healthcare provider priorities (transformation strategy, care model, national/regional agendas)

V.B.1.3. PROS/CONS Matrix and GO/NO-GO Criteria

Development of structured decision support document that includes the following information:

PROS AND CONS matrix
- PROS: Describe added clinical and operational value, innovation potential and strategic alignment, demonstrated performance and reliability of the innovation solution, integration feasibility and user acceptance (if applicable).
- CONS: Technical limitations, operational constrains, unsolved issues, regulatory and data protection constrains, economic burden, unclear ROI.
GO/NO-GO criteria
- Minimum performance thresholds achieved
- Acceptable risk level of unresolved issues
- Sufficient clinical benefit demonstrated during the pilot
- Feasibility of scaling within available resources
- Alignment with institutional priorities
- Market readiness and supplier roadmap credibility
Final decision options:
- GO – Continue collaboration, move toward procurement, or plan extended evaluation.
- Conditional GO – Proceed only if specific improvements or commitments are ensured by the innovation provider.
- NO-GO – Discontinue adoption due to lack of value, feasibility, or alignment.

Tips / Common Pitfalls

✅ Ensure independence: technical and clinical conclusions must be internal and unbiased. Using measurable indicators when possible.

✅ Evaluate the possibility to contact the innovation providers is there is a need for clarification of technical features.

✅ Include both process (development journey) and outcome (final solution) elements in the evaluation.

❌ Do not rely only on the innovation provider’s roadmap and data. Perform internal validation and verify evidence of progress and feasibility.

❌ Avoid overlooking regulatory roadmaps, which strongly affect feasibility and sustainability.

Outcome / Deliverables

Internal Technical and Clinical Evaluation Report, that contains technical, clinical, and innovation pathway analysis revised by managerial stakeholders.
Decision document (GO/Conditional GO/NO-GO) with rationale and recommendations.
Inputs for sustainability assessment and future procurement planning.

Resources

Need expert support for this step?

Browse Experts

Browse our directory of experts who can provide professional guidance.

Evaluation, Sustainability and Scale-up