Philips Marketing Claim Substantiation Study

During the verification of medical products in the course of a 510(k) process the FDA not only checks the technical documentation regarding compliance with applicable laws and standards, but also marketing claims that describe the performance of the product. If marketing claims are unsubstantiated, the FDA sends a Warning Letter similar to one sent for defective products.

In general, the FDA allows the use of comparative marketing claims, but expects that these claims are substantiated by data.

In the course of the development of a new mobile X-ray system the manufacturer consulted the Use-Lab GmbH in order to conduct a marketing claims substantiation study. The goal of this study was to achieve measurable results regarding the effect of new product features on the improvement of the workflow during surgical interventions.

Preparation

Based on the new product features, first ideas and claims were set up describing the desired increase in efficiency through the features. These claims affected a variety of aspects, for example improved communication between surgeon and medical-technological radiology assistant (MTRA), reduced intervention times and fewer unnecessary X-ray images.

On the basis of these topics a coding scheme was used to define further individual measurement points. This resulted in objective parameters (e.g., time for taking the first image, number of trials for a successful image, total time) as well as in subjective parameters (e.g., perceived workload, perceived usability). The entire study design was set up on these parameters, which had to be determined individually.

As a result, user tests with teams (for measuring the communication between team members) as well as user tests with individuals were necessary. For this reason two study designs were developed such that teams consisting of surgeons and MTRAs as well as individual users (MTRAs) could participate.

During the study the participants were asked to take several images with the X-ray system. However, the radiation source of the X-ray system was deactivated so that the participants were not exposed to any radiation. A cordless camera was installed that transmitted the images onto the screen integrated in the X-ray system was installed on top of the flat detector. An anatomical reproduction of the human sacral region with metal tubes in the typical projection planes served as a model for the study. Participants were tasked with positioning the camera so that an image could be taken with a more or less direct view through one of the tubes.

In total there were seven tubes which led to seven possible positions. Based on those positions several sequences were created. These sequences contained positions that needed to be approached only once as well as positions that needed to be reached several times. Participants had to remember the positions that had to be approached several times.

On the basis of a power analysis containing the factors supporting the claims the number of positions that needed to be approached individually was calculated. Overall the participants were asked to approach 25 individual positions. Thus up to 750 individual positions during 30 test sessions were created for later analysis.

A number of teams consisting of physicians and MTRAs who did not know each other, as well as individual MTRAs were invited to participate in this study. Besides testing the ease of operating the whole system, the tests with the teams also served to verify the claims regarding reduced miscommunication.

All participants were asked to make a run with deactivated new features. This situation corresponded to the actual state of using a device that is already established on the market. After a short break the participants were asked to perform further positioning tasks, but this time the new product features were active. To compensate possible adaptation and learning effects, the order in which participants used the version of the machine with the new features activated or deactivated was alternated between participants.

up

Data recording & analysis

Each test session was audio and video recorded for later data analysis. Furthermore, during each test session several data were recorded, e.g., the different times for task completion, the number of images taken until a given position was reached, the number of miscommunications, etc.

During the data analysis after completion of all tests further factors were identified by video analysis to revise missing or ambivalent data.

After data completion, various data analyses were run with a statistics program to test the significance of the previously identified hypotheses.

up

Results

The data obtained supported the significance of most of the previously identified marketing claims. The use of the new features led, among others, to following results:

  • The rate at which the X-ray system was moved in the wrong directing during positioning was reduced by 45%.
  • Miscommunications were reduced by 45%.
  • The subjectively perceived frustration level was significantly reduced (-48% for physicians, -72% for MTRAs).
  • The number of (unnecessary) images taken during repositioning was reduced by 55%.

Based on these results the manufacturer produced a white paper that reflects the summarized results and the performance of the new system.

You can find the complete study here

up

Cookies make it easier for us to provide you with our services. With the usage of our services you permit us to use cookies.
More information Ok