Key takeaways from this new article, for people blocked by the WSJ paywall:
"That letter cited “deficient practices” in five categories. In one of them, hematology, the problems found by inspectors posed “immediate jeopardy to patient health and safety,” regulators said.
The test that inspectors found Theranos kept doing despite the erratic quality-control results was a hematology test that measures how long it takes blood to clot, the people familiar with the matter said.
The test is an important part of the treatment regimen for patients who are at risk of strokes or have blood disorders...
People who have seen the Theranos inspection report said it seems to indicate the company ignored quality-control results for the blood-clotting test that fell short of its own criteria.
The results produced in the quality-control checks repeatedly deviated from the lab’s typical result by more than two standard deviations, those people said...
At Theranos, there were seven quality-control failures in a single day, said a person familiar with the report...
Dr. Hamill said any doctors who received prothrombin time test results from the Theranos lab during the six-month period shouldn’t rely on them and should have their patients retested as soon as possible. “Those results are not worth anything,” he said...
The federal inspectors also concluded that the Theranos lab in California used expired reagents, or substances added to blood samples to elicit chemical reactions, for some prothrombin time tests, according to the people familiar with the inspection report.
Those people said the report also found that the lab’s director didn’t have responsibility for the lab’s quality-control program and the lab’s quality-control manager was unqualified..."
EDITED TO ADD: Actually, the really key sentence is this one: "Theranos spokeswoman Brooke Buchanan said: “The PT/INR issues identified by CMS related to tests run on conventional equipment using venipuncture samples.” A CMS spokeswoman declined to comment." Because that means that these quality control problems were not due to Theranos' "revolutionary" finger-prick tests, they were from regular old venipuncture tests running on a standard machine that were giving wildly incorrect results, expired reagents, no QC in the lab, etc. Bad data is worse than no data. If they couldn't get old-fashioned labs right, no wonder they were having trouble with the finger-prick stuff.
That is rather scary. Quality control is a serious requirement to any competent lab. With physicians making decisions by results labs transmit, there is an inherent need of verifiable trustworthiness to the results.
A patient may be subjected to life altering medication or treatments, which is not a trivial responsibility that can be shrugged off with erratic processes or lack thereof.
A lab director's responsibility includes ownership of quality processes, reviewing abnormal results after the fact to see if proper processes were followed (such as communicating with the respective provider), reviewing financial success, and much more. To shrug off quality control and operate as if reputable is not just unethical, but incompetent. Such operations should cease immediately.
Disclaimer: I have experience in the medical lab industry.
Generally the FDA will give companies several outs to fix problems, they only bring out the handcuffs after several ratcheting steps (one of which includes bringing in a US Marshal and a team from the FDA, at your expense, to be at your business every day and tell you how to run your company).
The FDA does not have the power to jail people. Determining whether the state has sufficient reason to incarnate individuals is responsibility of the justice system. People go to jail when a judge and or jury recon thats justified. The FDA will make its case when it believes it has one.
Will anyone from a 10B startup go to jail ? Probably not, but we can schedule that for afternoon tea outrage.
Again, you're spouting Civics 101 answers here. I'm familiar with how the US justice system operates. The question isn't looking for that. The FDA obviously won't have an FDA Judge send people to FDA Jail after they've been arrested by the FDA Police. Literally nobody here thinks that.
But they do initiate investigations, and it is a crime to defraud or attempt to defraud the FDA. And if the FDA so chooses, they can forward their information to Justice and participate in the proceedings against the accused. They could provide the evidence of wrongdoing and increase the political pressure to act. They can do a lot to ensure that the state determines it has sufficient reason.
But who has standing to sue? Whoever suffered a stroke because their warfarin dose was too high or too low is dead, they cannot sue any longer. Whose dose was accidentally correct or whoever was lucky enough not to suffer a stroke did not suffer harm, so they can't bring suit either.
And Holmes and the board believed in good faith that their prothrombin test was accurate. No mens rea, no culpability. No prison for anyone.
Past experience with industrial accidents is that fines for safety violations resulting in death is a few ten thousand dollars. No one goes to prison ever, even if the omissions that caused the incident were grave. Here is a report on the fallout from an incident that caused four deaths and much head-shaking in the chemical engineering community: http://cenblog.org/the-safety-zone/2015/05/osha-fines-dupont...
With that background, your faith in the system is touching but unwarranted.
The government always has standing for criminal cases (the only kind that involve jail) because the offense is against society. The families of a deceased person has standing to sue in a civil case. Any one who had a test run by the company has standing to sue as well, even if there was no harm - they did not receive the product that was advertised, namely a validated test.
I doubt this will result in jail time for the principles, but they should be expecting lawsuits over this.
I would expect there are already multiple attorneys preparing to file class action lawsuits against Theranos (and Walgreens) on behalf of impacted lab test customers. The attorneys are probably searching for a suitable lead plaintiff right now. It's not necessary to prove that the patients were medically harmed by bad lab tests, just that they didn't receive the promised service. Theranos will probably settle to avoid an ugly trial, the lawyers will pick up a quick few million $, and the class members will get tiny partial refunds.
The federal agency tasked with that responsibility I would naively presume, or do you prefer trial by media ? And why, exactly, would someone that acted in good faith deserve prison ?
You don't have to mean for something bad to happen to get jail time. You're still responsible for your actions, intentional or not.
I'm not lawyer, so bear with me, but my lay understanding is that if it can be shown that they should have been aware of the quality and other issues regarding their results, they could be criminally negligent. Feel free to correct me.
I'm not a lawyer either. And being responsible for your actions certainly seems reasonable to me. But I cannot see how you could have acted in good faith if you were aware of quality and other issues regarding your result. That would be acting in bad faith.
This is speculation but...the company was founded in 2003....13 years ago and no product out yet...I have an awful feeling Holmes felt desperate, took on more than she could chew in order to compensate for the decade she lost, and now all the shortcut taking is catching up to her....
It's starting to look very ugly for what started out as something overreaching, and I think she will fight even harder to keep the house of cards from falling but pretty much this unicorn will be the first to go in 2016.
The key takeway I have from all this is if you are relying on Blood Tests for data that is critical to your health (as opposed to informative, educational, or personal interests) - it's probably a good idea, if you can afford it, to get results from three completely separate diagnostic systems - kind of like time with ntpd.
Not really. Normal labs that aren't aiming to be revolutionary ultra-profitable mega corporations will be fine.
Having a slightly more accurate result is useless, because there is far more biological variation in a person (from disease, the vagaries of physiology, compliance, pharmacokinetics, diet, the time of day, mental state, the season etc etc) than technical variation in a lab test. You would be optimising the wrong thing.
The only example I can think of where someone would probably be harmed by this is if they were undergoing some kind of a procedure with a bleeding risk, and the Theranos test said their clotting was fine when it wasn't, and they had a significant haemorrhage during the procedure. Someone will probably launch a class action lawsuit anyway.
I think that the Theranos board will realise that they need to get rid of Holmes in the next 6 - 12 months. Even if the tech actually works, having her there has become a liability.
Not practical for most patients in the real world. Insurers won't pay for redundant lab tests. Getting blood drawn multiple times is painful and logistically difficult. For patients with clotting problems just getting blood drawn presents some risk. And there often won't be 3 independent labs in the patient's area.
You could reduce your accuracy by doing three tests of the same blood sample (of course, exposing yourself to the risk that the blood was taken incorrectly, was stored in an incorrect vessel, was transported incorrectly, etc...).
In terms of practicality/Insurance - this would mostly be a function of how absolutely certain we are that the tests are accurate, and how damaging incorrect results would be. If we are 99.99% certain the values are correct, and incorrectly results wouldn't result in irreparable harm - then yes, not worth the investment in redundant lab tests.
But, if there is, say, a 5% chance I might die on an incorrect test result that has a 1% chance of happening - and that test is performed 12 times/year - then you can pretty quickly calculate the net-present-value of my life versus cost of test. (I.E. If I think my Life is worth $10mm, and the test normally costs $500 each time, or $6000/year, and I need to have this test done for the next 10 years, (assuming the value of my life doesn't drop over time, to make this simpler) - then each time I get the test there is a 1% chance of failure * 5% chance of death = 0.05% chance of dying as a result of each test.
The odds of me dying then, in the next 120 test = 100%-(chance-of-not-dying-120-times-in-a-row) = 1-.9995 ^ 120) = 5.825% chance of dying in this scenario using tests that cost $60,000. If I could reduce that to 1% (4.825% reduction) chance by spending an extra $120,000 (perform the test in two extra labs) - then my my investment of $120k would return 4.825%*10mm = $482,500.
You can work the numbers, assumptions, etc.. however appropriate - but for some combination of test costs, frequency, potential harm, and value assigned to life - there is quantifiable value that justifies performing the test multiple times (and, likewise, clearly circumstances in which, even if the test results are more likely to be inaccurate, not worth performing thee test).
Not having access to multiple labs is a real problem though.
That's not how healthcare economics works in the real world. No one can quantify the chance of death due to an incorrect test result precisely enough to calculate when it's cost effective to perform multiple redundant tests. The supporting data doesn't exist and no one is going to fund the research to figure it out. And what does 'incorrect' even mean? There is always some margin of error.
In real medical practice if a physician sees a lab result that is way out of range, or significantly different from the patient's previous result, or inconsistent with other aspects of the patient's condition, then she will order a repeat of the same test in order to confirm the results.
It's certainly how healthcare economics works. It determines what, if any tests should be run. It also determines how many parties/independent reviews should take place for high stakes tests such as tumor identification in carcinoma.
Other high-stakes scenarios in which the same tests are run multiple times are for high-economic cost events such as drug abuse, either in the workplace, or in sports.
You even note in your response, that, In real medical practice if a physician sees a lab result that is way out of range, or significantly different from the patient's previous result, or inconsistent with other aspects of the patient's condition, then she will order a repeat of the same test in order to confirm the results. - which is exactly what I'm suggesting people have done - only with a (presumably acceptable) time delay between when the first, and later tests are required. If the results of such tests were that clinical activity had to take place quickly, then you might not have the luxury of a delay, and in which case, multiple independent test vectors would be very useful - if they were economically feasible.
What people sometimes fail to consider - if the test was worth doing once - at, say, $600 - then why wouldn't it be worth doing (at presumably much higher confidence), three times at $100? And, if people agree, "OK, for $600 and $100 that makes sense" - they have to understand, those two numbers are completely arbitrary, and can only be understood in relationship to the value of the results. Whose to say there aren't a ton of tests today that are "$100" - that couldn't be done multiple times to be certain.
In fact, my friend was a lab technician in a private outfit. He quit when the boss said "no more running control tests before hospital tests. Its too expensive" So that meant they didn't even calibrate equipment/test for miscalibration any more! And the hospital is paying them for quality; they're delivering risk.
Not necessary. I worked in the medical device field for many years, writing software to control instruments that tested blood (serum).
The instruments can be trusted.
If a test is borderline, either the lab tech will manually retest it, or some instruments can be given business rules like "if result is on the threshold of good/bad, automatically rerun the test and provide both results to the physician."
We were well aware that the software we wrote affected real people's lives and acted accordingly. These idiots are only giving the rest of us a bad name.
If none of the three tests is calibrated and traceable to a standard you are just as well off throwing darts at a board and getting your numbers out of that. Take personal responsibility for your health and your bank account by throwing dice!
It's really funny how people have stopped hating on the FDA when Theranos is mentioned. In any other context the FDA is still worse than Satan and the pox.
If you're taking Warfarin, you have to get your INR checked every two to three weeks. It's really not practical to do that at three different labs every time. Also, this isn't some exotic edge case that only applies to a few people, but an extremely common medication. We're talking about huge numbers of patients here. If all their tests had to be triple-checked, the costs would be enormous.
See https://news.ycombinator.com/item?id=11243904 above as to why the cost issue is a risk assessment / return analysis. In certain scenarios in which potential harm is high from an incorrect diagnosis, it's worth the cost to perform the test completely independently.
Just starting to read up on this. Does anyone know if any of Theranos's customers used the lab results for ongoing medical treatment and suffered as a result of it?
In this case, money you should pay for bread at a supermarket can be considered a "paywall and discrimination against the poor".
There could be couple of reasons why instinctively we think about paying for bread as a mutually beneficial exchange of goods for money, and about paying for internet articles as a "paywall that discriminates the poor":
(a) Articles used to be free, and many (in other publications) are still free.
(b) There is a very little incremental cost to WSJ if a single article will be read for free by someone, as opposed to the supermarket example above where an incremental loaf of bread has a very real economic cost.
But that doesn't change the fact the WSJ article is a real good that cost a lot of money to produce and therefore should be paid for.
I've never seen so much scrutiny over a company before. Everyone is screaming "death to Theranos" at every opportunity. How much of that is purely balking over the high valuation? I have a feeling if they didn't raise so much money then journalists wouldn't be swarming this at all.
All of the critiques I see don't seem even close to show stoppers. No public peer reviewed studies? Some lab updates to match FDA regulations? Using some "unauthorized" third party equipment? This is a new company, why should these be the doomsday callings?
I have no idea if Theranos will succeed, but I can't shake the feeling that this media response is inflated due to the high valuation, and journalists are falling victim to the hype as much as the public, whereas this is a fairly normal health startup otherwise. It probably was a mistake to accept such a high valuation, so I don't think Theranos is totally innocent here.
It's also strange that a paywall article made it to the front page? I wouldn't guess there's a big overlap of WSJ subscribers and HN readers (lol conspiracy).
> How much of that is purely balking over the high valuation?[...]this is a fairly normal health startup otherwise
Absolutely not. Start-ups are supposed to fail early and fail often but when you're in the medical space you simply can't act entirely like a technology start-up. People's lives depend on many of these tests; you can't just go "well let's ignore the FDA and we'll take the slap on the wrist" like in other industries. Here they might deliver inaccurate test results that are then used to treat someone for a medical issue.
That can be deadly. Yeah many people are going crazy from the valuation and secrecy but there is plenty of legitimate scrutiny here especially if you read this article (I'm assuming you didn't because of your very generic Theranos comment versus addressing the issues in the article itself which sound grossly negligent, IMHO).
Let's see: Due to quality control “Those results are not worth anything,” Yet doctors were relying on them to adjust warfarin levels, where "Too much warfarin can cause fatal bleeding, while too little can leave patients vulnerable to clots and strokes, according to medical experts."
But the attention is because people can't stand high valuations?
Completely wrong. How far down do you have to read in this article to find Theranos' valuation?
All the way. It's not there. At all.
Nobody really gives a fuck about the valuation except valuation wonks. But a lot of people care a lot about this woman being a complete charlatan who is endangering public health.
A company offering medical services that pose "immediate jeopardy to patient health and safety" deserves scrutiny. If they knew that their results weren't good and didn't act on that information, why should they ever be trusted in the future?
This illustrates my point. How does holding a drop of someone's blood in a lab pose "immediate" jeopardy to patients? They're in the trial phase, right? These tests are not yet being used to determine actual medical outcomes. What's wrong with trial and error in the trial phase? It's like PCI compliance. Who cares if your company doesn't pass PCI certification the first time (no one? You're given a long amount of time to fix PCI compliance issues).
As the other comments have stated - this isn't about valuation. This is a medical company that can, and probably has already, endangered patients' lives through its failings.
There is a reason why certain industries are scrutinized so closely because of the effect they can have on people. Being in those industries, even as a startup, means you must follow the rules.
Or are you fine with your doctor using some "unauthorized" random equipment the next time you need critical tests or surgery?
Derailing, that's my biggest ":-/" with the Silicon Valley start-up "move fast and break things" culture.
If we as software engineers want to work on problems that matter, and not just dating apps and ad networks, we can't have a work ethic that encourages us to ignore troublesome laws and regulations and launch half-baked MVPs with a plan to maybe fix problems as they arise in future iterations. Sure, work smart, automate all the boring shit, but your automation has to be correct. Sure, focus on a MVP to get off the ground -- but make sure your product is viable, and correct across the minimum set of features.
Even when we are working quickly, sparsely, we should be laying good foundations for the future. We shouldn't be lazy conmen selling poisonous snake-oil.
"That letter cited “deficient practices” in five categories. In one of them, hematology, the problems found by inspectors posed “immediate jeopardy to patient health and safety,” regulators said.
The test that inspectors found Theranos kept doing despite the erratic quality-control results was a hematology test that measures how long it takes blood to clot, the people familiar with the matter said.
The test is an important part of the treatment regimen for patients who are at risk of strokes or have blood disorders...
People who have seen the Theranos inspection report said it seems to indicate the company ignored quality-control results for the blood-clotting test that fell short of its own criteria.
The results produced in the quality-control checks repeatedly deviated from the lab’s typical result by more than two standard deviations, those people said...
At Theranos, there were seven quality-control failures in a single day, said a person familiar with the report...
Dr. Hamill said any doctors who received prothrombin time test results from the Theranos lab during the six-month period shouldn’t rely on them and should have their patients retested as soon as possible. “Those results are not worth anything,” he said...
The federal inspectors also concluded that the Theranos lab in California used expired reagents, or substances added to blood samples to elicit chemical reactions, for some prothrombin time tests, according to the people familiar with the inspection report.
Those people said the report also found that the lab’s director didn’t have responsibility for the lab’s quality-control program and the lab’s quality-control manager was unqualified..."
EDITED TO ADD: Actually, the really key sentence is this one: "Theranos spokeswoman Brooke Buchanan said: “The PT/INR issues identified by CMS related to tests run on conventional equipment using venipuncture samples.” A CMS spokeswoman declined to comment." Because that means that these quality control problems were not due to Theranos' "revolutionary" finger-prick tests, they were from regular old venipuncture tests running on a standard machine that were giving wildly incorrect results, expired reagents, no QC in the lab, etc. Bad data is worse than no data. If they couldn't get old-fashioned labs right, no wonder they were having trouble with the finger-prick stuff.