04 October 2017

I come to bury the GRE, not to praise it

I’ve seen a few graduate programs announce that they are not going to require students submit GRE scores any more. These announcements are widely met with praise. The GRE has minimal predictive value in long term grad school success, and it is biased against a lot of groups. And the costs stops a lot of people from applying to grad school.

Interestingly, at the start of last year, the dean of our graduate college announced that several programs were being required to add the GRE to their admission requirements. This was imposed on at from outside the institution at the state level. I can’t remember if it was UT System or the THECB.

Full disclosure. When I became the graduate program coordinator of our master’s program, I pushed and got our department to start requiring the GRE. My rationale at the time was that this was the “industry standard.” We wanted our students to go into doctoral programs, and we reasoned that we would be helping students pave the way for doctoral work by having them do it sooner rather than later.

Also, I was reacting to students who would come in the day before classes started and say, “Can I be a grad student?” At the time, there was no application deadline. And students who did that tended not to persist in the program. So requiring the GRE forced students to plan ahead, not go a grad school because there was nothing good on television that day.

I have since come around to see the many problems with the GRE. But I don’t think our department would be allowed to get rid of it, seeing how many departments were force to require it.

But this is something I think about.

The GRE tried to solve a couple of problems. It failed to solve them, but those problems still exist. And I don’t know how to solve them. The problems are:

  • Grading policies vary wildly across institutions. (See this blog post.)
  • People interpret the same grades in different ways depending on the institution’s perceived rigour and prestige. (See this blog post.)
  • Recommendation letters are usually uniformly glowing.
  • People tend to trust recommendations “in network” from people they know either personally or by reputation.

Students from famous universities who have rubbed shoulders with famous professors and can convince them to send a form letter get deep advantages in grad school acceptance. In other words, we end up selecting for students for grad school who already have a lot of “social capital.” If we want to diversity science, this is not the way to go about it. Diverse students come from diverse institutions, as Terry McGlynn has noted.

In theory, the GRE could have acted as a leveler for the playing field. It didn’t. But the problem it could have tackled is one that we still need to tackle. What can help level the playing field for students against “prestige”?

Related posts

What grades should look like
The “Texas transcript” is a good idea, but won’t solve grade inflation

External links

Students, Rejoice — Standardized Testing May Soon Be Dead

02 October 2017

The little known ways neurons communicate


I just looked in the introductory textbook we use for general biology, and it provides a good explanation of chemical neurotransmission between neurons. If you’ve studied basic biology, a diagram like this probably looks familiar:


I was impressed that the textbook mentioned for chemical neuromodulation. The presynaptic cell releases chemicals across the synaptic cleft, but the receptors don’t open ion channels. Instead, they interact with metabotropic receptors that cause biochemical cascades inside the neurons. These cause any variety of slow, long-lasting effects.

But there are at least three more ways that neurons can communicate.

Third, there are gasotransmitters: small, short lived gases that are made on the spot and zip through cell membranes like they weren’t even there. There are at least four different gases that do this (reviewed in Wang 2014). Nitric oxide is the best known in nervous systems.

Fourth there are electrical synapses. Neurons can be connected by gap junctions, which create pores in the membranes that allow ions to flow freely from one neuron to another. Consequently, an action potential spreads from one cell to the next as though they were one big cell. These were first described in crayfish, by the way (Furshpan & Potter 1957).

Fifth, there are ephatic signals, where the electrical potentials generated by spikes in one neuron create electrical potentials in adjacent neurons, even with no synapses or gap junctions between them. It’s just a spread of electrical activity. This was described at least as early as 1940 (Katz & Schmitt 1940), but I am ashamed to admit I had never heard of this until a few years ago (Su et al. 2012), even though the original effect was shown in the 1940s using motor neurons in crab legs! And learning about those was a big chunk of my doctorate.

I often find myself griping about how many people assume that brains work like computers. And I think part of that is because signalling by chemical neurotransmission seems very computational. I wish more people knew that there are at least four other ways that neurons can interact and influence each other. Maybe they wouldn’t be so quick to think that downloading our brain activity into computers is going to be easy.

References

Furshpan EJ, Potter DD. 1957. Mechanism of nerve-impulse transmission at a crayfish synapse. Nature 180(4581): 342-343. http://www.nature.com/nature/journal/v180/n4581/pdf/180342a0.pdf

Katz B, Schmitt OH. 1940. Electric interaction between two adjacent nerve fibres. The Journal of Physiology 97(4): 471-488. https://doi.org/10.1113/jphysiol.1940.sp003823

Su C-Y, Menuz K, Reisert J, Carlson JR. 2012. Non-synaptic inhibition between grouped neurons in an olfactory circuit. Nature 492: 66-71. https://doi.org/10.1038/nature11712

Wang R. 2014. Gasotransmitters: growing pains and joys. Trends in Biochemical Sciences 39(5): 227-232. https://doi.org/10.1016/j.tibs.2014.03.003

Picture from here.

29 September 2017

My K-index


Matthew Hahn reminded me of a paper that proposed the “Kardashian index” for scientists. I don’t like the paper. It demeans outreach by implying that a high score (author Hall suggested more than 5) is an attention seeker who contributes little and should get back to the lab. There’s some pretty crummy sexist overtones in the choice of “Kardashian” as a descriptor. There was much criticism of this paper when came out, which I’m not going to rehash here.

But I didn’t know there was an online calculator. (Hat tip to Genome Biology.) And yes, I’m curious enough to enter in my numbers. My score was 17.46; the highest of anyone in Hahn’s Twitter thread by quite a ways.

I like trying to do outreach, and I know I do niche research. So this is not surprising to me.

If you love science, love methods sections


Mensh and Kording (2017) have a new paper on scientific writing. It’s very good. I agree with most of their advice. But not this.

You should also allocate your time according to the importance of each section. The title, abstract, and figures are viewed by far more people than the rest of the paper, and the methods section is read least of all. Budget accordingly.

No. Do not skimp time spent on your methods sections.

I get where this advice is coming from. It’s the same sentiment that has lead some journals to put their methods section at the end, or to stuff parts of papers away in online “supplemental information.”

But we read papers for lots of different reasons. I read lots of papers that are only tangentially related to me out of curiosity. But when there is a paper that is in my field, that I need to understand, I dig deep into those methods sections.

I’ve run into so many cases where something that looked like a solid finding looked very shaky once you realized how the data were collected. While Mensh and Kording are right that few people read the methods, it neglects that those who do are going to be the most intense and critical readers.

A recent feature in Nature showed that weak detailing of methods was leading to irreproducible results (my emphasis).

In one particularly painful teleconference, we spent an hour debating the proper procedure for picking up worms and placing them on new agar plates. Some batches of worms lived a full day longer with gentler technicians. Because a worm’s lifespan is only about 20 days, this is a big deal. Hundreds of e-mails and many teleconferences later, we converged on a technique but still had a stupendous three-day difference in lifespan between labs. The problem, it turned out, was notation — one lab determined age on the basis of when an egg hatched, others on when it was laid.

The article give multiple examples of how hard it is to standardize methodologies, but how important it is to achieving consistent results. This older Drugmonkey post, makes a similar point.

The methods section is where the rubber meets the road in terms of actually conducting science. If you don’t get that methods section right, you’re wasting the time of people who come afterwards.

References

Mensh B, Kording K. 2017. Ten simple rules for structuring papers. PLoS Computational Biology 13(9): e1005619. https://doi.org/10.1371/journal.pcbi.1005619

Lithgow GJ, Driscoll M, Phillips P. 2017. A long journey to reproducible results. Nature 548: 387–388. https://doi.org/10.1038/548387a

External links

The most replicated finding in drug abuse science

28 September 2017

Paying out of pocket


Anne Madden asked:

Academic scientists, how many of you have contributed significant out of pocket funds (or fam. $) to make your science happen?

 My newest paper cost me, personally, at least $5,919.21.

Every month for five years, I drove from Edinburg to Beach Access Road #6 on South Padre Island. Google Maps says that 92.2 miles, so that 184.4 miles round trip. Five years is 60 months, and the going rate for mileage reimbursement in Texas is $0.535.

92.2 miles × 2 × 5 × 12 × $0.535 / mile = $5,919.21.

And I know there are months I went more than once, so that is a conservative estimate. I also ate lunch on every one of those collecting trips. So maybe another $600 on top of that.

That project also involved family money, because my mom bought a new shovel when I was on the beach collecting and the one I was using broke.

It’s probably good if I don’t do this calculation very often.

I also paid out of pocket for this year’s American Society for Parasitologists meeting in San Antonio. The meeting was practically in my backyard (only a four hour drive; that’s close in Texas), so was relatively cheap (a drive to San Antonio is much less than a plane ticket). It was close to the end of the fiscal year (ours starts 1 September), and there is rarely travel money in the budget by then. Plus, there’s just less paperwork.

That said, I know I have it better, and I have reached a point of financial security where I can “opt out” of dealing with the torture that is university purchasing and reimbursement. Others are not so lucky. Here’s Holly Bik (lightly edited):

Serious proposal: If we want more minorities / first generation students to stay in science, we need to fix the travel reimbursement pyramid scheme.

I just found out that my university, University of California Riverside, can only prepay my conference registrations with a paper form (and it takes 3 weeks). Admins can’t pay anything travel-related with credit cards (they don’t have them). Seriously. It’s 2017. Everything is online.

I’m a first generation college student with $150,000 in student loan debt. And now UC Riverside wants me to pay more than $5,000 out of pocket for my work-related travel. For this summer’s conferences, I’ve probably paid more than $200 in credit card interest while waiting for reimbursements (money I don’t get back). Travel reimbursements varies across universities – some are pretty good. But UC Riverside is probably one of the worst I’ve experienced to date.

Sometimes there are the “perfect storms” of conference, workshop, etc., travel invites. And these are so important for early career people. So if you have money, you can travel for work (and suck up out of pocket expenses). And you meet people, build a network, have successful career. But if you are saddled with debt, you may forgo important opportunities because you just can’t eat up those travel costs. Your career suffers.

The biggest irony is that UC Riverside is proud of us first generation grads and faculty, but institutional bureaucracy works against us in horrible ways.

Institutional purchasing is terrible. And sometimes I think it’s bad on purpose, to drive people like me into paying for thing myself that the university should pay for, just because I don’t want to deal with the perpetual hassle and headaches of trying to fill out forms and get reimbursed.

Update, 29 September 2017:  Tara Smith contributes to the discussion.

From Bik’s thread, some places seem to be able to front costs–why can’t that be universal? It seems like a small thing when you have money, but for many struggling academics it’s the difference between “making it” and leaving the field.

Related posts

Indie spirit
Who paid for my open access articles?

External links

The high cost of academic reimbursement

27 September 2017

Scientific societies need to compete on services


There’s an editorial in the journal mBio asking members to publish in society journals.

The editorial contain some nice data outlining the changes in the publishing scene. But the argument the editorial advances sound like special pleading.

Just as the prefixes “Nature” and “Cell” seem to bring gravitas to many journal titles for some scientists and represent implied signals for authors and readers about the quality of the papers that they publish, journals published by professional scientific societies should carry the same authority. After all, they have a long tradition of authoritative leadership and management and are edited by some of the most accomplished scientists in their fields. Professional societies provide legitimacy to the journals they publish. When an author submits a paper to a scientific society journal, or when someone reads a paper published in a scientific society journal, they can be assured that the journal is legitimate and has a decades-long track record of quality.

This paragraph may be right that journals from societies “should” have authority. But you cannot assert authority or credibility. Credibility is determined by what other people say or do. The editors should be asking themselves the hard question of why they lost that credibility.

For instance, my reaction is similar to one I had for Evolution Letters. Let’s do a price check. It costs at least twice as much publish in the society journal mBio ($2,250-$3,000) as PeerJ ($1,095). It’s not clear to me what that I get shelling out all that extra cash.

The editorial tries to claim societies offer superior editorial services.

Scientific society journals are managed and edited by scientists actively working in the fields covered by the journals. ... Although there are certain challenges in using academic editors (12), they bring the experience, expertise, and authority that enables professional societies to refine their missions and set the standards of their fields as they evolve.

This one is tough, because all I can say is that I have published with many journals, and I have not seen consistent differences in the editorial services between those run by scientific societies and those that are not. A Twitter poll suggests society journals may have a very slight edge.

The editorial ends with a very generic one that publishing in a society journal “helps the society.” That may be, but people will only stick with sub-par services out of sentiment for so long.

Society journals are carriage makers that see the Model Ts on the road. And they don’t know how to adapt. Their only argument is, “But the horse is a noble animal. You like horses, right?” If scientific societies don’t figure this out quickly, they will be relegated to a tiny little niche in the academic publishing industry.

Related posts

The problem is scientists, not publishers


References

Schloss PD, Johnston M, Casadevall A. 2017. Support science by publishing in scientific society journals. mBio 8(5): e01633-17. https://doi.org/10.1128/mBio.01633-17

26 September 2017

Microsoft Academic: Second impressions


By happenstance, I thought of Microsoft’s imitation of Google Scholar, Microsoft Academic Research, yesterday. I reviewed it years ago, but hadn’t thought of it much since then. I wondered what happened to it.

Quite a lot, as it turned out. The original website was decommissioned, and the name was shorted to Microsoft Academic. Version 2 launched in July 2017. I thought it was worth a new look.

Sadly, the second look is not much more promising than the first one.

One of the changes is that you can create a profile of your papers. That could be good. I’ve found profiles in other similar sites to be kind of useful. Okay, you have to create an account. No problem, I do that all the time... and you hit the first oddity.

Weirdly, you must use another site to set up your account. You can’t just give an email address and pick a password, like pretty much every other website on the planet. You have to use Twitter, Google, Facebook, Microsoft, or LinkedIn.

I thought, “My institutional ID is handled by Microsoft, and I use that to log in to Office 365, so that should work.” Nope. So I logged in with Google.

I discovered a possible reason why Microsoft Academic won’t let me use my institutional email as I start building a profile. It asks for institution, and snootily insists, “An affiliation must be selected from the list of suggestions.” Except that, according to it, the University of Texas System consists of just five institutions, not fourteen. And my university was not one of those five.


Building out a profile was weird, too. I thought that this next gen scholarly database would support ORCID, so you could enter one number and have it gather all your publications. Nope.

Microsoft Academic seems to identify authors by some mystic combination of name, institution, and... something else. For example, it considers the Zen Faulkes who was at the University of Melbourne a different author than the Zen Faulkes at the University of Texas-Pan American.


So you have to go in and “claim” your papers from however many random ways Microsoft Academic has parsed your name. I have a very distinctive name, and my papers were split into something like ten different authors. I cannot imagine how many different ways publications might be split if you have a common name. Or if you changed your names.

For some of my new papers, I found them by searching the database for the title. I added them to my profile by clicking the paper’s hyperlink, clicking “Claim,” then going to another page and clicking “Submit claim” again. It seemed to be a lot of clicking.

The profile lists “top papers,” but the metric Microsoft Academic uses to determine “top” papers is not clear. It’s not citations, because the number of citations in my list of papers are: 11, 28, 17, 9, 16.

Maybe the profile has a few weak spots, but good coverage of the scientific literature might make it valuable. I searched for “Emerita benedicti” (on my mind since the publication of my newest paper last Friday). That gives 15 hits in Microsoft Academic, but over 2,300 in Google Scholar. But if I search for that exact combination of words in Google Scholar, I am still left with over 32 hits, more than double Microsoft Academic’s yield.

Microsoft makes much of Academic’s “semantic search,” so it may be that I will find it more useful as I try other, more complex searches, rather than simple things like looking for a simple species name.

The home page for all this provides a customized dashboard with a calendar of scientific conference, research news, recent publications, and recent citations of your papers (not visible in the screenshot below).


Google Scholar gives you a couple of alerts on its home page, a sparse approach that leave you with no doubt as to what its job is: Google Scholar is a search engine. It’s not quite clear what Microsoft Academic wants to be. The home page of Microsoft Academic feels more like it wants to be a science news feed; more the science section of Google News than Google Scholar.

Perhaps the kiss of death in all this is that practically everything on this site feels like it’s moving through molasses. It’s sloooooooow. I spent a lot of time looking at a screen like this, waiting for it to populate:



Even while writing this blog post, I got a “You do not have permission to view this directory or page” error message when I tried to go to the home page. Google Scholar feels like it’s using telepathy in comparison. (Update, 26 September 2017: A bot attack may have been responsible for this slow performance.)

I will keep trying Microsoft Academic for a while to see if I learn more. But this project is now at least six years old. And darn it, it still feels like a clunky beta version, just like it did back in 2011, not a modern 2.0 version of an academic search engine that it says it is.

Related posts

Microsoft Academic Research: First impressions

External links

Zen Faulkes’s  Microsoft Academic profile

24 September 2017

Paying to publish and Poynder

Richard Poynder and Michael Eisen got into it on Twitter over the weekend over open access publishing. Poynder wrote:


My view is that PLOS legitimised a deeply-flawed business model: pay-to-publish.

Hm. The problem is that many journals used “pay to publish” before PLOS journals came along. They were called “page charges.” You can still find many journals with page charges that are not open access. Cofactor has a list here.

These seem to indicate that asking scientists to bear some of the cost of publication is not inherently problematic. At least, I certainly don’t recall any serious discussion about them as deeply flawed. There probably should have been. But people accepted page charges as a normal, routine part of some corners of academic publishing. Saying PLOS legitimized that model is questionable.

PLOS ONE revolutionized academic publishing. But what was revolutionary was its editorial policy of not screening for “importance.” That lead to it publishing a lot of papers and generating a lot of money. It was through that combination that PLOS ONE paved the way for many imitators, including bad journals (documented in Stinging the Predators).

To me, the bigger problem is that “pay to publish” is very often equated – wrongly – with “open access.” The business model used to support publishing is not closely related to whether people can freely read the paper.

External links

Journals that charge authors (and not for open access publication)