Complacency in science

I have finally read Galbraith’s Short history of financial euphoria, which Alain Aspect suggested to me during a random dinner chat a few months ago. It’s nice: it’s the first time I understand something about finance. And it triggered a concern about academia.

In finance as well as in academia, people often fall into euphoria over something that is, by all rational standards, rather worthless. In my field of research, for instance, the latest craze is the following process:

1. Write down a new version of some criterion that tests that “something is quantum” (a new Bell inequality, a new test of contextuality, a version of Leggett-Garg…); the simpler — the more trivial — the better, because of point 2.
2. Find a couple of friends to do an experiment for you. Better if they have been running their setup for ages and have exhausted all the serious science that could possibly be done with it, because they will be more than happy to learn that their old machinery can still be used to perform “fundamental tests”. Moreover, since your test is simple and simple quantum physics has been tested to exhaustion, you have no doubt that the experimental results will uphold your theory.
3. If you can, present it as “the first step towards [a big goal]“. Never mind that it is rather the last use of a setup that has made its time (I refrained to use “swan’s song”, because the last song of the swan is supposed to be the most beautiful; the last concert of an 80 years old pop star would be more appropriate a metaphor). If you can’t invoke the future, present it as “the conclusive proof of [some quantum claim]“. Never mind that the claim is usually always the same, namely, that results of measurements are not pre-established, that there is intrinsic randomness, or however you want to phrase it. Also never mind the fact that there cannot be a “conclusive claim” every month.

The euphoria mechanism is entertained as follows:

• The big journals (Nature at the forefront) prefer to publish tons of poor science rather than risking and losing a single real breakthrough. So, if someone claims to have solved “the mystery of the quantum” (the general readership of Nature finds quantum physics mysterious), better take them seriously.
• In turn, people notice that “if you do that, you publish in Nature”. Since “that” is not that difficult after all, it’s worth while going for it.
• Once you have published in Nature (or Science or…), you are hailed as a hero by the head of your Department, by the communication office of your university, by the agencies that granted you the funds.
• Put yourself now at the other end, namely in the place of the one who would like to raise a dissenting voice and reveal the triviality of the result. All the legitimate instances (peer reviewed journals, heads of prestigious Departments, grant agencies, even popular magazines and newspapers!) are against you. Isn’t it “obvious” then that you are only venting your jealousy, the jealousy of the loser?

So far, the analogy with financial euphoria is clear. I guess (though I have not studied the statistics) that the speed of the crash is also analogously fast: it happens when some of the editors of the main journals take a conscious decision of having “no more of that”, because they realize that there is really nothing to gain. The rumor spreads that “refereeing has become tough”; the journals are accused of having become irrational since “if they accepted the previous paper, why they refuse this one” (while it’s one of their few moments of rationality).

And the consequences? The same too, but fortunately without criminal pursuits, despair and suicides. The very big fish get out unscathed: either their science is really serious (that is, they have invested only a small amount of their scientific capital in the euphoric topic); or their power is really big (that is, they have invested only a small amount of their political power in backing the euphoria). The opportunists will try to follow the wind as they should, and will be forgotten as it should. Those who face uncertain destiny are the young fellows, who were doing serious science when the euphoria caught them at the right time and the right place. Because of this, they have been raised to prominence. Somehow, all their capital is invested in that topic. Will they be able to find their way out and continue doing serious science? Or will they end up teaming with their buddies, set up a specialized journal for themselves and publishing there until their old age? If one day you find me as the founder of a journal called “Nonlocality”, please wake me up.

Happy Easter!

Measuring uncertainty relations

In the space of two weeks, two works appear in Nature Physics about measuring uncertainty relations. In the first, an experiment is actually performed to test (and, needless to say, verify) the validity of an uncertainty relation which applies to more situations than the one originally devised by Heisenberg. In the second, it is proposed that the techniques of quantum optics may be used to probe modifications of the usual uncertainty relation due to gravity. Now, to have finally a tiny bit of evidence for quantum gravity, this would really be a breakthrough!

Faithful to my principle of not doing “refereeing on demand”, this is not an unrequested referee report: in fact, I have only browsed those papers, certainly not in enough depth to make judgments. The authors are serious so, by default, I trust them on all the technicalities. The question that I want to raise is: what claims can be made from an uncertainty relation?

An uncertainty relation looks like this:

[something related to the statistics of measurements, typically variances or errors] >= [a number that can be computed from the theory]

which has to be read as: if the left hand side is larger than 0, then there MUST be some error, or some variance, or some other form of “uncertainty” or “indeterminacy”. Let me write the equation above D>=C for shorthand.

Now, let’s see what a bad measurement can do for you. A bad measurement may introduce more uncertainties than are due to quantum physics. In other words, one may find D(measured)=C+B, where B is the additional contribution of the bad measurement. It may be the case that your devices cannot be improved, and so you can’t remove B. Now, the second paper proposes an experiment whose goal is precisely to show that D(measured)=C+G, where G is a correction due to gravity. Obviously, much more than the mere observation of the uncertainty relation will be needed, if someone has to believe their claim: they will really have to argue that there is no way to remove G and not because their devices are performing poorly. The problem is that there is always a way of removing G: a bad measurement can do it for you!

Indeed, a bad measurement may also violate the uncertainty relation. Let me give an extreme example: suppose that you forget to turn on the powermeter that makes the measurement. The result of position measurement will be systematically x=0, no error, no variance. Similarly, the result of momentum measurement will be systematically p=0, no error, no variance. In this situation, D(measured)=0. Of course, nobody would call that a “measurement”, but hey, that may well be “what you observe in the lab”. To be less trivial, suppose that the needle of your powermeter has become a bit stiff, rusty or whatever: the scale may be uncalibrated and you may easily observe D(measured)<C.

So, a bad measurement can influence the uncertainty relation both ways, either increasing or decreasing C.

Now, there are reasonable ways of getting around these arguments. For instance, by checking functional relations: don’t measure only one value, but several values, in different configurations. If the results match what you expect from quantum theory, a conspiracy becomes highly improbable; and indirectly it hints that your measurement was not bad after all. For instance, this is the case of Fig. 5a of the first paper mentioned above.

Still, I am left wondering if the tool of the uncertainty relation is at all needed, since by itself it constitutes very little evidence. Let me ask it this way: why, having collected enough statistics for a claim, should one process the information into an “uncertainty relation”? The information was already there, and probably much more of it than gets finally squeezed into those variances or errors. OK, maybe it’s just the right buzzword to get your serious science into Nature Physics: after all, “generalized uncertainty relation” will appeal to journalists much more than “a rigorous study of the observed data”.

Map your project

“I live in a house with garden in North America”. What is preposterous about this sentence? If you speak about your garden, you would not expect the whole of North America as a location. You’d expect something like “I live in a house with garden in the District D, Town T, Canada”.

Well, that is the level of introduction that you get in many papers and scientific presentations: let me make up one. “Quantum computers will have capacities beyond those of any classical computer. Here, we study how a bi-exciton decays in the quantum dots that our group has been trying to fabricate for 10 years with moderate success” (the last half is usually phrased differently, but everyone knows what that means). The point I am making here is: there is a world between the grand dreams of the field and the specific research topic one is dealing with. Following up on a point of a previous post: it is very useful to take some minutes to locate your project on the map of science, with a gradual zoom:

Continent – Nation – Town – Street – House

or, if you prefer:

Grand field – Big challenge within the field – Approach to the challenge – Specific technique [here is "the specifics of my boss"] – My project

Example:

Quantum computing – Experimental quantum coherence – Artificial atoms, Quantum dots – Self-assembled GaAs quantum dots – decay of the bi-exciton

Once you have established this map, here is my advice on how to structure a 20 minutes (10 slides) presentation, according to the circumstances. Take it with flexibility of course

• Undergrad project: 1-1-3-2-3 [your project is probably an incremental step in your supervisor's field, so it's better to take a close focus; but two or three slides on the bigger picture are necessary]
• PhD Thesis defense: 1-2-2-1-4 [your project is more relevant, so it should contribute at least a bit to the "big challenge", if not to the grand field]
• Generic conference: 1-3-3-2-1 [people should remember that "you work in that challenge", they will forget the details]
• Conference of your grand field: 0-2-3-3-2
• Conference of your big challenge: 0-1-2-3-4
• Specialized workshop: 0-0-1-3-6 [here is where people really care about your technicalities]
• Grant defense: 5-3-0-0-2 [fine print will be lost, but you have to show that you are doing everyday progress]

A last word: contrary to geography, starting from your project you may zoom out in different ways. For instance, a study on quantum dots may also be seen as belonging to material science, or to quantum optics, rather than to quantum computing… Some people like to bring up all those maps at once: it’s a dangerous option, because it may confuse the audience on your motivation and also give the impression that you are trying to “play up”. The safe option consists in choosing one of the possible maps (the one that your public may like most) and stick to it.

Theorists collaborating with experimentalists: a tentative guide

This post is triggered by some of my students. I could give them my own advice directly, but I prefer to write it here because they may benefit from other valuable feedback. The issue at stake is pretty specific: the role of theorists collaborating with experimentalists. The background is my own, limited experience, the classification is as crude as it gets, the analysis proposed here is not the only possible one and is not even always suitable — so, let’s start!

It seems to me that theorist can contribute in four ways to experiments:

1. Vision: come up with a new concept. It can be very general (say, quantum optics; quantum computing; quantum cryptography) or more specific (say, squeezed states; measurement-based quantum computing; quantum repeaters), but in any case it opens paths that were unexpected before. You won’t find these ideas planned in research proposals: they “happen”. I don’t suggest you base your career on the hope of having such a flash of genius (though you should be ready to recognize it, if it comes).
2. Tools: develop the necessary theoretical understanding. To continue with the examples above: develop the formalism of quantum optics; invent fault-tolerant architectures; provide the formalism to do security proofs.
3. Proposals: suggest that a given experimental setup can be used to observe some interesting effect, and do a rough first feasibility study.
4. Specific description: take a real experiment and describe the physics, to the point where your theory matches the data.

As you can understand, these categories are not tight: for instance, some proposals make it directly into an experiment (nowadays, this is the route of choice to publish your proposal in Nature).

OK, now: here you come and have to decide in which direction to go. You may want absolutely to study a type of experiment: then you need to know what people in that field would appreciate. Or you may want to spend your life (say) inventing proposals: then you need to assess which experimental field is ripe for those. In both cases, the crucial insight for me is: there is a strong correlation between the role of a theorist and the stage of development of the experimental field:

• At the beginning of a field, proposals are really important; they are also the fast way to celebrity (example: Cirac-Zoller first proposal of a logic gate in quantum computing). If you are up for a longer-term investment, go for tools even at this stage: your effort will be appreciated with a lag, but when the moment comes, you are seen as a pioneer and you are among the few of can really make a difference (example: the work of Norbert Lutkenhaus on unconditional security of quantum cryptography).
• Once the field is mature, proposals must become really relevant (some physics journals can be seen as a cemetery of irrelevant proposals). Rule of thumb: if no experimentalist cares about your proposals, with overwhelming probability you are not the misunderstood genius, but just the delayed fellow (though, of course, exceptions are possible). At this stage, tools are the most appreciated contribution of theorists: if you develop them, you can choose to keep contact with the labs, or you can take the way of mathematical physics and develop them for their own sake. Both ways are serious; my advice is, but whichever you take, keep an eye on the other.
• When a field is so mature that even the tools are fully developed, there is little left to do other than specific descriptions (this is the case, for instance, of quantum optics per se, i.e. aside from possible applications in quantum information). Specific descriptions of experiments are tricky. First, you need to check if the experimental group “needs your service”: some groups, for instance, have developed their tools so well, that their experiments are “textbook experiments”, which means that anyone who can follow a textbook can do the theory. Now, if your help is welcome, it is normally very welcome. Since there few people able to do that (compared to the mass of people contributing to the cemetery of proposals), you will be noticed among the experimentalists and may be asked for various collaborations. This is great for a PhD and, if you have such competence, you would do well in keeping it alive for the rest of your career. But you have to do something else as well, if you want to get a position: you need to show that you are also capable of original work (see why in a previous post).

Peer review

Recently, I have submitted four papers to a conference with different co-authors. After the peer review process, one was accepted for a talk, the three others for a poster. I do not copy all the reports here because it would be boring, but just the marks we received to the question “is this worth a talk”, ranging from +3 to -3. You will see a pattern emerge.

Paper 1 (the one that was accepted) had three reviewers: marks 3, 2 and 1

Paper 2 had three reviewers: 2, 0, -2

Paper 3 had two reviewers: 1, -2

Paper 4 had two reviewers: -3, 2 (the first reviewer, having noticed a few typos, mentioned “poor right up” [sic] as one of the reasons not to consider our submission).

Do you see the pattern? No? Look more closely… YES, you have got it: peer reviewing is random number generation

Technical corollaries:

(1) With little post-processing, any correlation with the content of the paper can be removed for papers 2-4.

(2) Paper 1 is special, not because there is no spread, but because the average is not centered around 0. This bias is robust and can be eliminated only by suppressing buzzwords.

How to convey the wrong message

Last week, two undergraduates I know managed to convey the wrong impression about themselves and their work in a remarkably instructive way. They have learned from their (ultimately harmless) mistakes. Maybe someone else can learn too.

Case 1: a student presents the progress in his project. He starts by stressing that the title has changed because the initial project proved too ambitious. The rest of the talk is a review of some schemes for atom cooling: nice and clear, but covering pretty well known material and leaving important schemes aside. You are listening to him. What do you conclude?

Here is a pretty reasonable analysis: the student did not match the expectations of the supervisor, so the initial ambitious project was tuned down to something hardly more than a review of literature.

Here is the truth: the incompetent fellow, if anyone, is the supervisor (myself), who did not evaluate correctly the difficulty of the initial project. The student is doing very well, he has done much more than a simple review in terms of calculations and simulations. The review was simple because I asked him to use graphs and pictures instead of equations; it was incomplete because the goal is to describe a real ongoing experiment, not to review the whole field of atom cooling!

Where did the student fail? Here are some hints:

1. Opening the talk by stressing that the focus of the project has changed was the wrong thing to do: it conveys the message that something has gone wrong. Incidentally, the initial topic was pretty similar and nobody would have noticed the change.
2. The student embarked in the usual idiotic “atom cooling is a fundamental whatever-not in modern physics, interesting both for our understanding of nature and for applications” and failed to mention the REAL motivation, which is the description of an experiment which is really happening two floors below.
3. The talk was too simple. One must be clear of course, but if it is a research project, one must devote one or two slides to show off — I mean, to convey what has been done.

Case 2: after a few months of research under the direction of a post-doc, a student is asked in the office of the professor (a Singaporean Chinese, which matters for what follows). The professor starts by asking “What have you learned of this field so far?” and the student replies “Nothing much really”.

Here, you don’t have to be a professor to guess the rest. What is more astonishing is the background story.

The truth is that the student had learned a lot and was aware of it. But she feigned ignorance in order to give the professor the chance to explain things from the beginning and learn from his insight! She thought that, by acting this way, she would show how eager to learn she is and how much she appreciates the wisdom of the professor.

This attitude, to this extreme, can only be found in students who have been formed in the Confucian style. But a milder form happens everywhere: for instance, when a professor asks in a lecture “Do you know this or do you need a reminder?”, almost always the students ask for the reminder (maybe there it’s also a trick to slow down the pace and trick the professor into not covering too much new material). In the context of master-to-disciple relationships, it may have its value (though I personally hate it). But if you are going into research, you have to show what you know and admit what you really don’t know: both false humility and false confidence will be detected and signal the end of your application.

An ongoing experiment in sociology of science

Two months ago, Pusey, Barrett and Rudolph posted on the arXiv a paper with the title The quantum state cannot be interpreted statistically. It generated a lot of hype, and also serious interest. As for myself, I did not finish to understand it (my latest attempt is at the bottom of this series of comments), but this is not the matter now.

Two days ago, the same Barrett and Rudolph posted a new paper on the arXiv (with two other co-authors). The title: The quantum state can be interpreted statistically.

What’s going on? There is no mystery: in the first paper, the no-go theorem was proved under some assumptions; in the second paper, one of the assumptions is removed and an explicit model is constructed. Scientifically, this is nice: it clarifies the conditions, under which a statistical interpretation can be constructed.

This post is not meant as a criticism of the scientific content of those papers — I don’t do refereeing on demand, and even less by blog. I am just alerting those interested in sociology of science. Indeed, in the coming week or so, it will be very instructive to monitor the reaction of the scientific media to this new paper. The claim as stated in the title would deserve the same hype as the previous one (if you care about A, both the propositions “A is false” and “A is true” carry the same importance). My prediction is that, while there may be discussions in scientific blogs, the journals will not pick up the story this time. Anyway, let’s not waste more time by discussing all possible scenarios: we’ll discuss post factum on the one that will actually have happened.

Advances in foundations?

Yesterday I attended a talk by Daniel Terno. It was about one of his recent works, re-examining the so-called delayed choice experiment and its implications. It was a well-delivered talk with an artistic touch, as usual for the speaker. Berge Englert, who was in the audience, made a few historical observations of which I was not aware: notably, that the idea of delayed-choice dates back to 1941 [K.F. von Weizsäcker, Zeitschrift für Physik 118 (1941) 489] and that Wheeler conveniently omitted to quote this reference in some occasion (I don’t know how he knows this, but Berge knows a lot of things). I learned something during those 45 minutes

I went home mulling on the end of the exchange between Berge and Daniel. Berge stressed that he doesnot understand why people still work on such things as local variable models. He added that, in his opinion (well informed as usual), all the foundational topics have been discussed and settled by the founders and the rest are variations or re-discoveries by people who did not bother to inform themselves. Daniel replied that he basically agrees, but since these days many people are excited about closing loopholes, he argued that these discussions are relevant to the community. I think both are right, but they also forgot an important point.

I agree with Berge that there is no “fundamental” reason to keep researching on local variables and their alternative friends (contextuality, Leggett-Garg…). Journals like Nature and Science are filled with such experiments these days; but this fashion and consequent flow of publications is not driven by controversy, nor by the desire of acquiring new knowledge, because everyone knows what the outcome of the experiment will be. It is a most clear form of self-complacency of a community. I agree with Daniel that there is some need to put order in the clamor of claims, so a clean analysis like the one of his paper is very welcome.

However, I think that there is a reason to close those loopholes: device-independent assessment! If quantum information tasks are to become practical, this is a really meaningful assessment. Experimentalists (I mean, the serious ones) do worry about side channels. If they could go for device-independent assessment, their worries are excluded by observation. But to reach there, you need to close the detection loophole.

I also think that the notion of device-independent is a genuine advance in foundations. I side with Berge when it comes to all the debates about “wave vs particle”, “indeterminacy”, “incompatible measurements”… On such topics, yes, the founders settled pretty much everything. But I don’t see how people immersed in those terms of debate could have anticipated an assessment of non-classicality made without describing the physical system at all: that is, without saying that “it” is an electromagnetic field in a box (Planck), or a magnetic moment in a magnetic field (Zeeman and Stern-Gerlach).

Now, you may wonder why device-independent does not receive so much public praise and excitement as the other stuff. I don’t know, but several reasons may contribute to this situation:

* The debates on waves and particles have percolated into the general educated public. Since there is no “clear” explanation available (there cannot be, if by “clear” we mean “understandable in everyday terms”), these educated people think that the problem is still open. Scientific journalists, for instance, pick up immediately every paper that hints at some advance in wave-particle blabla — I suggest they should always consult Berge before writing enthusiastic nonsense. The idea of device-independent is too novel to generate such an excitement.

* None of the great North-American prophets of the church of larger Hilbert space (i.e. the quantum information community) is preaching for device-independent. The topic is being pushed from some places in Europe, where they have a network (principal investigators: Antonio Acin, Nicolas Gisin, Serge Massar, Stefano Pironio, Jonathan Barrett, Sandu Popescu, Renato Renner) and from Singapore (Artur Ekert and myself).

* Device-independent theory is tough (you need to compute bounds without assuming anything about your system, using only the fact that you observed some statistics); experiments are even tougher (you need to close the detection loophole at the very least, as for the locality loophole, either you close it too, or you need a good reason to argue it away). So it’s a sort of “elite” topic, which does not gain visibility from mass production — yes, a constant flow of papers, even if most of them are deemed wrong or pointless, does contribute to the impression that a topic is hot and interesting.

And finally, the most powerful reason: I am neither Scott Aaronson nor John Baez, so nobody reads my blog

A tale of 2011

Many things happened in 2011, of which I can only be thankful. I wanted to consign one to record, which may otherwise be missed, because it is about a “failure” — or better said: a beautiful reaction to a disappointing realization.

Starting in August 2010, a student of mine, Thinh, had been studying a new class of protocols for quantum cryptography, inspired by a previous work. By April, he had managed to define the key mathematical objects to very general scenarios. This was his Final Year Project (FYP), which was awarded as “Outstanding” by the university. A few months later, together with Lana (post-doc), we prepared a paper and submitted to Physical Review Letters (PRL; for the unaware: one of the most prestigious journals for physics).

When the referee report came, the tone was expected: “good work but not of enough broad interest” — very common nowadays for quantum cryptography. The referee stressed how he/she liked very much our generalization, i.e. Thinh’s result. With a few modifications, we could have had the paper published in Physical Review A (PRA; a very good journal still, edited by the same society; a Tier 1 journal in NUS, for the sake of the bureaucrats who care about these classifications).

However, one of the small comments of the referee caught our attention: we realized that the family of protocols we had considered was uninteresting! In a nutshell, these protocols collect a lot of information, but then discard much of it and rely on the rest. Why should one do so?? In other words, all that we did was correct and even elegant, but the object of our study was sort of pointless.

Now you see the alternatives we were facing: (1) skip this awareness under the carpet, do the modifications suggested by the referees and submit to PRA, with quasi-certainty of being accepted; (2) forget about this paper and write rather a technical note, explaining why these protocols are not interesting, to be sent to a very specialized (i.e. less visible) journal. For me, there was no doubt that (2) was the correct course, but I let Thinh and Lana decide — and I am very proud to say that they took the right decision The paper has duly been re-written and is under consideration in a specialized journal of our field.

Now comes the scary part of it. I told this story to several friends working in the academic world, over coffees or lunches or other informal meetings. Many of them, especially the younger one, were astonished: “Wow, you guys are so honest! I know many who would never had dropped the chance of publishing in a Tier 1 journal”. For myself, I am sure that Thinh and Lana have made a bigger step in their career by choosing the right course: if you keep your standards high, Tier 1 publications will come.

Happy New Year!

 

Everyone is speaking of it, part II

The more I hear about this result, the more I fear that the media have picked it up only because they misread the meaning of the title… Let me explain what is done there as simply as I can. I’ll let the reader decide if they think the media could have understood this

Let L (lambda in the paper) be a list of deterministic instructions of the type {“Measurement A –> give result a”, for all measurements}. Since quantum states do not predict deterministic results for all measurements, a single list is trivially inadequate. But there is a very natural way to generate randomness: just pick different lists {Lk} with probability pk each. So, the model is:

Quantum state of a single object <–> {Lk, pk}.

What the paper proves is that no two quantum states can share any list: the set of lists with probability non-zero uniquely identifies a state. In other words, giving the possible lists, or even just one of them, is equivalent to describing the state…

… for a single object! Indeed, Bell’s theorem proves that not only a product of lists {La,pa}x{Lb,pb}, but even a single product list {LaxLb, pab} cannot describe entanglement. So, lists just don’t seem to do the job. Personally, I can’t believe that the randomness of one quantum object comes from a list, when we know that the randomness of two quantum objects cannot come from a list.

In the same vein, I have a small problem with the logic of the proof. One constructs a family of product states, which should be obviously described by products of lists, and measures them by projecting on a suitable family of entangled states, which… which… wait a second: how does one describe entangled states in that model?? It seems that the closest attempt was Spekkens’ toy model, which reproduces many nice features of quantum physics, but unfortunately not (guess what?) the violation of Bell’s inequalities. Maybe the contradiction exploited in the proof comes from the fact that there is no description of entangled states in a model with lists?

That being said, this paper does add something for those who still were trying to believe in lists as explaining quantum randomness — and the more this idea is shown to be inadequate, the better

Note added: I was convinced that this post misses the point, but it triggered some nice follow-up; so please read the subsequent thread of comments: the “truth” may be at the bottom — or in the exchange