Clinical trials are crucial to modern healthcare industries, and information technologies have been employed to improve the quality of data collected in trials and reduce the overall cost of data processing. While developing software for clinical trials, one needs to take into account the similar patterns shared by all clinical trial software. Such patterns exist because of the unique properties of clinical trials and the rigorous regulations imposed by the government for the reasons of subject safety. Among the existing software development methodologies, none, unfortunately, was built specifically upon these properties and patterns and therefore works sufficiently well. In this paper, the process of clinical trials is reviewed, and the unique properties of clinical trial system development are explained thoroughly. Based on the properties, a new software development methodology is then proposed specifically for developing electronic clinical trial systems. A case study shows that, by adopting the proposed methodology, high-quality software products can be delivered on schedule within budget. With such high-quality software, data collection, management, and analysis can be more efficient, accurate, and inexpensive, which in turn will improve the overall quality of clinical trials.
Clinical trials (CTs) are measures to evaluate the safety and efficacy of a medical device or drug. CTs are crucial to modern healthcare industry and usually take substantial resources and time to complete with four major parties involved: the sponsor, the government agency, the participating hospitals (or sites), and the patients (or subjects) [
Today, almost all fields, including the healthcare industry, are benefited by high-speed computers and information technologies [
More precisely, eCRFs are considered the replacement for paper-based CRFs while eDCs focus on two things: (1) data are captured remotely and (2) the whole lifecycle of data is always in electronic forms. It means that each site must have an eHR system to build eDC [
In this paper, a “Clinical Trial System” (CTS) is defined as a software system to handle trial data; therefore, a CTS includes data-related software, for example, eDC or eCRF, but excludes the rest, for example, CT management or workflow control. The three core components of a CTS are the data collection module, the data exportation module, and the backend database system. A CTS, eventually, exports data for statistical analysis by statisticians with professional statistics software, which is excluded in the current definition of CTS because it requires expertise completely different from software development. From the software developers’ point of view, the main complexity of developing a CTS comes from the data collection system rather than data analysis system which requires skills from statisticians.
To deliver a quality software product as complicated as a CTS on time within budget, a predefined software development methodology (SDM, or so-called process model) should be followed. However, adopting existing purpose SDMs to develop a CTS requires the development team to disregard some fundamental guidelines of the SDM because all CTS share some unique properties (described in details in the following section).
The rest of the paper is organized as follows. Section
Modern CTS works not only as a vehicle at front end for collecting data but also as a complete system which can be as complex as any commercial software. To maintain data integrity, a CTS usually applies constraints such as simple value range check and complex cross-form value derivation. For reasons like data auditing, CRFs may also include nonnumerical fields like hand-drawn pictures (e.g., the position of ulcer), photographs (e.g., photographs before/after operations), or texts (e.g., doctor’s comments). In addition, complicated flow branches can be defined in the protocol (e.g., subjects with different conditions/test results may require different flow path to fill additional/different CFRs).
Figure
Example eCRF for “Assessment of Ulcer Wounds”.
Figure
Three commonly used CTS architectures.
Figure
Figure
A process model is defined as “
The planning spectrum of SDM [
The properties that distinguish clinical trials from other application domains are described with the example (shown in Figure
The process of FDA clearance to market a Class III medical device.
(1)
(2)
(3)
This pathway is complicated with certain patterns. Due to the involvement of government agencies, the pathway is different from most processes of software development in other domains such as financial, industrial, or insurance. Based on the discussion above, a CTS has the following unique properties.
CTS can be developed by generic software development tools or CTS-specific tools, for example, Oracle Clinical [
Comparison of CTS development by generic and specific tools.
Specific tools | Generic tools | |
---|---|---|
Costs for acquiring tools | High | Low |
Costs for qualified engineers | High | Low |
Costs for adding special features | High if not supported | Low |
End user learning curve | Relatively high | Low |
Can CTS utilize software already acquired? | Depends | Yes |
Any SDM to follow | Depends on vendors | No |
Is the project manager required to know tools in depth? | Yes | No |
The purchasing fee for general development tools is relatively lower than the cost of a specific tool. The cost for hiring engineers or project managers familiar with a specific tool like Oracle Clinical is high since such people do have specific knowledge. In Figure
In general, developing a CTS with generic tools costs less than with CTS specific tools. Although free software is available, for example, EpiData [
This subsection reviews core concepts of the current SDMs in plan-driven methods, risk-driven methods, and agile methods as illustrated in Figure
Plan-driven methods include traditional SDMs like
The risk-driven spiral model is an evolutionary model with four regions:
Unlike the plan/risk-driven approaches, agile methods give up the concepts of “phases” and focus on the flexibility of the methods. Agile methods represent a group of light-weight methodologies including
It is obvious that some of these focal values may not be proper for CTS development. For instance, for focal value (2), CTSP8 shows that a partially working CTS can hardly provide benefits to the CT. For focal value (3), contracts between the developer and customer may not be necessary if they are in the same organization, for example, a pharmaceutical company. For other cases where the CTS is developed by a software consultant firm (with CTSP1~6), negotiating a contract agreed by both developers and customers is important. For focal values (4), since all CTS development follows the same process, as shown in Figure
Furthermore, some of the 12 principles of agile development are not suitable for CTS development by nature. For instance, the third principle “
Table
SDMs and the CTS properties that make the SDM improper for CTS development.
SDMs | CTS properties that make the SDM improper for CTS development |
---|---|
Plan-driven | CTSP1, 3, and 6~11. |
Risk-driven | CTSP1 (the risks of CTS development are different from risks discussed in risk-driven methods) |
Agile methods | CTSP1~8, 10 |
Due to CTSP3~5, engineers may not obtain the CTS specification at an early stage of the project, which is a necessary condition for plan-oriented SDMs. On the other hand, agile methods focus on prototypes, rapid values, and constant meetings, and so forth, disregarding that engineers can take advantage of the fact that every CTS development shares the same patterns and properties. To design an SDM that fully covers the CTS properties and takes advantage of both plan-oriented and agile methods, one needs to carefully review the protocol evolution. Hence, the new SDM for CTS development naturally resides in the middle of the spectrum as shown in Figure
The state diagram of the protocol evolution at hospital and sponsor site.
To handle the unique patterns and properties of CTS development, a new SDM is proposed, as shown in Figure
The proposed CTS SDM in UML activity diagram.
The first phase,
Different from most current SDMs, the sponsor and engineers can design the system architecture, for example, Figure
CTSs with different architectures may have totally different software components. For instance, if trial data can be retrieved from hospitals’ eHR, then the CTS can be designed without an interface for manual data input. If data must be manually entered, a standard 2-tier database system should suffice for a CT without complex semantic checks. Otherwise, a 3-tier system is more appropriate for hosting those checks. Along with the
The
The second phase focuses on system analysis and design. Major work products include the
For cases in which the protocol is approved by the FDA and the participating hospitals before CTS is completed, the sponsor and engineers have two choices: (1) reject qualified subjects before CTS is completed or (2) allow subjects’ registration and prepare an alternative plan to collect data before CTS is ready. The most popular alternative plan is to use traditional paper-based CRF to collect data. Alternative planning requires the latest amended protocol and the
For the analysis/design paradigm, either a function-oriented or an object-oriented approach can be adopted since both approaches are well established in the field of software engineering [
The third phase, the construction phase, focuses on developing a functional CTS data collection subsystem. Major work products are the
To implement a single eCRF, as shown in Figure
During implementation, engineers may receive conditional approval message “Data spec. changed” from FDA or IRB. In such cases, all activities must be interrupted (dashed rounded box in Figure
If FDA approves the IDE after all construction cycles are completed, then the subject enrollment may begin with the integrated CTS data collection subsystem without any delay. Otherwise, the CTS
The last phase is the exportation phase because the CTS requires exporting data for statistical analysis. The development of data exportation subsystem needs not to be executed in the previous phase based on CTSP1 and CTSP12. Major work products are the
The construction cycle is similar between the third and fourth phases, but the fourth phase focuses on data exportation components. The construction cycle in both phases can be implemented in parallel if no complicated dependent rules exist. In the fourth phase, engineers work with statisticians on data exporting specifications and develop, test, and integrate software components into CTS. All supporting documents are supposed to be created at this time such as the
There are two backward flows in the proposed SDM from the construction phase to the first and second phases in order to preserve the flexibilities. The development is parallel since the CTS constriction is based on components. Components which are not affected by a certain amendment can stay in the third phase. This approach enables the proposed SDM to have the advantage of incremental model with agile flexibilities. As illustrated in Figure
In summary, the proposed SDM covers all thirteen CTS properties with the following features: (1) architecture design is performed at the first phase, CTSP12; (2) the development of CTS is split into two well-defined subsystems, CTSP1; (3) all phases are designed to be iterative and incremental, CTSP2~5; (4) engineers do not need to be involved in protocol development or colocated with sponsors, CTSP6~7; (5) there are two concrete milestones at the end of third and fourth phases (delivery of data collection subsystem and delivery of integrated CTS), CTSP8~10; (6) the limited time for development is optimally utilized, CTSP1; (7) experiments for engineers and PMs are not necessary conditions.
The proposed SDM has been applied on three CTS developments and one of them (for the Phase-I study of a new silicon human dermis developed in Taiwan) is discussed in this section to demonstrate the feasibility and effectiveness. The government regulations for CTs in Taiwan are very similar to those in the USA with a similar pathway as shown in Figure
The study initially involved four participating hospitals without a common eHR; therefore, a 2-tier architecture similar to Figure
For comparison of cost, all three CTS developments adopting the proposed SDM are considered against the previous two CTS developments. The development cost was measured by accumulating all costs (for project manager, system designer, software developers, database developers, testers) divided by the number of eCRF. The average cost per form (as shown in Figure
One of the many advantages of this new SDM is that phases and work flows can be followed easily so that PMs can coordinate the CTS development effectively and efficiently. It is therefore not necessary to have members from the sponsor side to assume the role of PM; engineers or primary investigators can also be the PM, too. On the contrary, the CTS experiences of PM may become a major roadblock in a conventional SDM. As aforementioned, the risk of CTS development is high if it adopts agile methods and is supervised by an inexperienced PM. Similarly, the CTS experiences of engineers can reduce the risk of failure, but it is no longer a necessary condition because the activities are well defined and, as addressed earlier, the major risks of CTS development come from its unique properties rather than the experiences of its developers.
The results of the case study show that the proposed SDM outperforms conventional SDMs in many ways. First, the proposed SDM has the advantages of both the plan-oriented methods and agile methods. On one side, activities and phases in the proposed SDM are well planned, defined, and easily followed as in plan-oriented methods. On the other side, the proposed SDM maintains the flexibility to respond to specification changes as in a typical agile method. Second, the proposed SDM specifically discards the guidelines/principles suggested by both plan-oriented methods and agile methods which are unrelated to or violate CTS properties. Third, time is used as efficiently as possible in the new SDM to reduce the number of subjects needed to use the CTS alternative plan. Fourth, the flexibility of adopting a specific SDM for individual construction cycle is maintained. Engineers are allowed to adopt a familiar SDM, for example, the waterfall model, for a particular construction cycle for data collection and exportation to develop individual software components. Finally, the flexibility of a CT with multiple CTSs in different architecture is preserved. It is not necessary to have a CTS for all sites in the proposed SDM. Each CTS with different architecture can be implemented independently as long as the data exported to statistical software are considered acceptable by statisticians.
Based on the experiences of developing CTS with the proposed SDM, a few guidelines are proposed to reduce the risk of project failure. First, engineers should work closely with both the sponsors and the statisticians. Before the third phase is initialized, engineers must have a clear idea of what data will be collected and what data format will be used. At the exportation phase, engineers should provide data files in the formats the statisticians demand. For example, data in the same form with different visits can be exported differently (into one file or file per visit). Next, it is important to make sure that the engineers follow the proposed SDM and understand the unique properties of CTS beforehand. The sponsor cannot assume that the engineers who may have never participated in CTS development are familiar with the pathways as illustrated in Figure
Handling late updates is always a challenge; that is, protocol changes after CTS is online. It may be rare but can happen. If the missing data are available in hospital records, the currently online CTS can still be used to enroll subjects. In the meantime, engineers should restart the activities in the second phase to retrieve necessary data. If the amended protocol requires data that are unavailable in hospital records, the data collected from enrolled subjects should be discarded and a new CTS development should start all over again.
Over half a century ago, software engineers recognized the importance of SDM and proposed several primitive and pioneering models like the code-and-fix and stage-wise model [
As for future works, more case studies of CTS developments can be performed with the proposed SDM. Research teams with recourses can apply the proposed SDM with an existing SDM on a same CTS development at the same time as the experiment group and control group, respectively, to compare the results.