Skip to content

How to Write a Publishable Social Scientific Research Article: Exploring Your “Process”

2014 July 16
by Richard N. Landers

If you ever talk to an artist about how they create whatever it is they create, whether that is acting, pottery, or blog articles, you will eventually begin to talk about their “process.” The process is whatever technique people use to go from a default state (e.g., bored and watching TV) to a state of creation.

A person’s process is really about two things: it is about motivation and technique. If you’re an actor, you must motivate yourself to throw yourself entirely into the emotional and behavioral state of a character.  You must also do so well.

When writing, recommendations about process focus largely on writer’s block.  As a person writing a research article, with a decent process, you should never have writer’s block.  The reason is that scientific writing is a highly technical, by-the-books sort of task.  You don’t need to be inspired, and you shouldn’t wait to be inspired. In fact, writing scientific articles while inspired is the leading cause of bad scientific articles. Instead, you need to be methodical. With sufficient technique, motivation becomes less relevant.

With that in mind, here is my process, which is really all about technique. I use this process in my own writing, and I recommend it to all of my graduate students. Following it step by step helps to avoid extensive re-writing, which is the worst nightmare of many a graduate student.

Step 1: Identify your audience and eventual publishing venue. If writing a thesis, this will be your committee. If writing a journal manuscript, this will be the journal’s reviewers.

As a graduate student, audience can be a tough one. I think, by default, graduate students tend to think of the audience as “people who know more about my topic than I do.”  But it is a bit more specific than that.  People that are asked to be reviewers for particular journals are asked for one of four reasons.

  1. They may be asked by connection.  For example, the editor has collaborated with the reviewer in this area before and leans on that relationship to get a timely review.
  2. They may be asked by guilt. For example, the editor is a little desperate for reviewers, so s/he asks an early career person, like a grad student or first-year PhD, or any random person off of the journal’s prior reviewer list.  Such folks are generally scared to turn down any opportunity/responsibility that comes their way, so they almost always say “yes” even if they know little about your topic.
  3. They may be asked by association.  For example, if you base your paper largely off of a particular other researcher’s work (i.e., their name is cited in your paper a lot), that person may be asked.
  4. They may be asked by reputation.  This is where most people assume reviewers come from, but it is probably the least common source in practice across journals.  When you submit to high-tier journals, though, you’re more likely to get this sort of reviewer.  In this case, a known expert in the topic of your paper reads your paper.

In terms of reviewer quality, the guilt and connection groups are your worst nightmares, because they are completely unpredictable.  The best way to write for these reviewers is simply to use clear, unambiguous prose. Even if those reviewers are skimming through your paper, they should understand what you’re trying to say.  I’ll return to this point in a later Step.

The association and reputation groups are the ones you’re really writing for.  They know what you’re talking about, and they want to know what you are contributing and why.  People in these two categories generally don’t read your article word for word. The reason for this is that they already know most everything you’re saying. You’re citing literature they’ve read before, describing ideas they already know. Instead, they want to know what unique message you are delivering with your paper. So to determine this, many experts read topic sentences only and briefly skim the rest of each paragraph, taking time to read in deatil only when they are unsure what point you’re making.

This is really the secret of scientific writing. You make the major points you are trying to get across to support your argument in the topic sentence of each paragraph. The sentences that come after that point only serve to support that topic sentence. In scientific writing, you should not make callbacks to previous points unless absolutely necessary, and you (usually) should not use transition sentences between paragraphs.  Each paragraph should be a self-contained little island of information - a major point, following by information to support that point.

When reviewing, reviewers identified by association and reputation will expect this format and use it to get to the meat of the paper as quickly as possible. Scan, scan, scan – oh look, a paragraph that makes a unique point that I’ll read more closely – scan, scan, scan.  If your reader can’t tell what point you’re making in a paragraph, they may skip it anyway and then miss whatever you were trying to say.  If you don’t follow the topic-then-support pattern, your paper will be more difficult for an experienced scientist to read, and a difficult-to-read paper is a quick path to rejection.

If this sounds different than what you learned in English class, you’re right.  Narratives rely on a poetic flow of ideas expressed eloquently, although not necessarily clearly (depending upon the author’s intent).  For maximum rhetorical impact, the author of a narrative may leave out important details to encourage you to think a certain way, only revealing them later to maximize the impact on the reader.  In contrast, scientific writing relies on predictability and fully supported points.  You must always make your points up front and then provide supporting details.  This takes a dramatically different approach than writing a story – which is what we’ll get to next.

Step 2: Write a Skeleton Outline matching the publishing venue’s expected article structure.

Once you’ve identified the sort of expertise and expectations of your reviewers, you’ll want to start to meet those expectations.  The first step in this process is laying out an outline that matches the required format of your intended publication target.  These vary widely.  In Psychology, you might need to write a paper in APA, or you might not.  Check the journal’s website and find out.  For our example here, I’m going to assume that you’re writing an manuscript for a journal requiring APA style.  That means the next step is creating this outline:

TITLE: TBD

  1. (Introduction)
    1. (Introduction to the Introduction – Less than 4 pages)
    2. Heading 1
    3. Heading 2
    4. Heading 3
  2. Method
    1. Participants
    2. Materials
    3. Procedure
  3. Results
  4. Discussion
    1. (Introduction to the Discussion)
    2. Limitations
    3. Practical and Theoretical Recommendations
    4. Conclusion

That’s it – an entire paper.  Note that a few sections are surrounded by parentheses.  That is because these headings are not “real” headings – they will not go into your paper.  Instead, they are implied by the structure of your paper.  All other headings (the bolded ones) will eventually appear, word for word, in your actual paper.

There are a couple of special sections worth noting at this point.

The intro to the intro is 4 or fewer pages that come immediately after your title but before any other headings.  The intro to the intro is the most important part of your paper.  It should set up all major arguments that you plan to make in your paper and all major terms that you will use in your paper.  If your reviewer can’t figure out exactly what you’re researching and why by the end of this section, you will get a negative review.

The intro to the discussion is the second-most-important section.  Folks in the connection and guilt reviewer groups are going to rely on this section to tell them exactly what your paper contributed and if it was important.  This is a terrible shortcut for them to take, but it is unfortunately pretty common.  Ensure you make all of your hard-hitting conclusions in this section, right up front. Don’t save the best for last.

That’s it at this stage – ensure your skeleton outline matches the journal’s expected format, and you’re ready to move to Step 3.

Step 3: Create a Summary Outline by filling in the outline with verbatim headings, summaries of major points, and target page counts.

This outline is, at this point, pretty fluid.  You want to keep the overall structure of the skeleton outline, but you may add or remove headings as needed.  You might, for example, decide you need to have headings in your results section because you have a lot of results to get through.  You might decide to split up theoretical and practical recommendations because you have a lot of recommendations.  This is the stage to make those decisions.  Here’s an example:

TITLE: An Amazing Exploration of Writing Scientific Papers

  1. (Introduction)
    1. (Introduction to the Introduction – Less than 4 pages)
      1. Discuss how popular paper writing is
      2. Discuss how more research is needed on paper writing
      3. Describe how we’ll be testing paper writing
    2. Paper Writing Theory (2 pages)
      1. Paper writing theory is complicated
      2. Paper writing requires training
    3. Paper Writing Training (2 pages)
      1. Training is very important to improve paper writing
      2. Training delivers paper writing theory to those in need
      3. Past research has supported the use of training to improve paper writing
        1. H1. Being trained in paper writing will improve paper writing.
  2. Method (3 pages)
    1. Participants
      1. Participants were some random wahoos from the Internet
      2. Summarize demographics here
    2. Materials
      1. Describe paper writing intervention
      2. Describe outcome measure (noting inter-rater reliability)
    3. Procedure
      1. Describe procedure
  3. Results (1 page)
    1. We did an independent-samples t-test, and we found a big effect which was also statistically significant.
  4. Discussion
    1. (Introduction to the Discussion – 2 pages)
      1. Paper writing training totally worked!
      2. Describe implications for paper writing theory
    2. Limitations (1 page)
      1. We didn’t actually conduct the study, so that’s limiting
      2. The participants were random wahoos from the Internet
    3. Practical and Theoretical Recommendations (2 pages)
      1. We need more work on paper writing theory
      2. We need more brilliant blog articles on paper writing
    4. Conclusion (1 page)
      1. Things are better now because of my paper

This outline should be shared and worked on collaboratively with any co-authors, although I recommend it be written initially by the primary author alone. In doing this, all writers will know exactly which points have been made by any given point in the paper, but the primary author will be responsible for the overall vision and progression of what is written. This is helpful for several reasons, most especially, if I’m working on the discussion and coauthors are writing everything else, I won’t need to see their actual writing to know what has already been said.  This supports a coherency of writing in multi-author papers that is difficult to achieve otherwise without hiring a professional editor post hoc.  This also increases the overall speed of article production, since less back-and-forth coordination is required later (minimizing what is called “team process loss”).

By the end of this stage, all writers should agree that the flow of major ideas supports the points being made.

Step 4: Create a Detailed Outline by replacing major point summaries with verbatim topic sentences.

In the next stage, each author should update the summary outline by replacing general concepts with actual topic sentences.  For example, the intro to the discussion might be changed to this:

  1. (Introduction to the Discussion – 2 pages)
    1. As hypothesized, the training of paper writing skills produced a strong, positive effect upon paper quality scores as made by expert raters.
    2. This study’s major contribution to paper writing theory is our integration of the effects of training on existing theoretical frameworks.

When creating a detailed outline, you should try not to add or remove any paragraphs that you’ve already agreed upon in the previous Step. Sometimes it is necessary, but if you completed the previous Step correctly, it will be minimal. It happens most often when writing topic sentences – you realize that something you summarized before is more complicated than you initially believed.

Topic sentences are the only point at which there should be “transitions” in your paper. A topic sentence can refer to a previous topic sentence if two points are closely tied together. However, you should avoid this unless absolutely necessary so that the paper is easier to reorganize later, if necessary (for example, by reviewer request in a revise and resubmit).

Once all authors have finished their sections, the complete outline should be reassembled.  Because the topic sentences are where the “story” of your paper is told, you should be able to read your entire outline of topic sentences and come away with a complete picture of everything you argued, everything you did, and everything you contributed. If a reviewer were to take everything in this version of your outline at face value (i.e., if they didn’t care about previous research supporting your points), your outline alone should form a compelling argument for what you did.

After all authors have reviewed all contributions and agreed that the topic sentences make a compelling and fully supported argument, you move on to the next Step.

Step 5: Convert your outline into a paper by adding supporting points to each topic sentence.  Try not to change any topic sentences at this point.

In scientific writing, actually writing your paper is the easiest step! At this point, you have a very specific blueprint. You’ve finished all the hefty thinking work already! Thus, the final step is simply to add supporting details, with plenty of citations, for each of your points. The key in this stage is that every sentence you write – every word – should be written to support the topic sentence of the paragraph you are currently writing.  Don’t worry about additional planning, or additional points, and remember to ensure that each paragraph is self-contained.  If you do end up changing a topic sentence, or adding or deleting one, make a comment (Word’s reviewing tools are great for this) to alert your co-authors of the change, just in case they based some of their points on yours.

Step 6: Check for logical consistency and voice in the completed paper and edit as needed.

In the final step, all contributions from all authors should be combined into a complete paper. First, review this paper for consistency - if anyone changed a topic sentence, you may need to revise the flow of the overall paper. This is where self-contained paragraphs are useful; if you need to move anything, you don’t need to worry about breaking the flow of the paper because you can see exactly which points fall where. Especially important here are term definitions – if you introduced a term in Paragraph 10 and then move Paragraph 12 to become Paragraph 8, remember to check that you didn’t use that term from Paragraph 10 before defining it. Second, review the paper for voice – ensure that there are not obvious “breaks” between different authors where the style changes dramatically. It is typically the primary author’s responsibility to resolve such discrepancies.

That’s it! If you complete all of these steps, you will have a must better chance at ending up with a logical, well-argued, social scientific paper suitable for publication. Well done!

Share

Is Outrage Over the Facebook Mood Manipulation Study Anti-Science or Ignorance?

2014 July 2
by Richard N. Landers

ResearchBlogging.orgBy now, you’ve probably heard about the latest controversy coming from Facebook – a researcher internal to Facebook, along with two university collaborators, recently published a paper in PNAS[1] that included an experimental manipulation of mood. Specifically, the researchers randomly assigned about 700,000 Facebook users to test an interesting causal question: does the emotional content of posts read on Facebook alter people’s moods?

This is an interesting and a compelling way to study that question. Research to this point has focused on scouring pre-existing news feeds and inferring emotion from posted content. This has led to some other well-known Facebook research about the Facebook envy spiral, in which people post exaggerated accounts of the positive events in their lives, which leads reading users to become jealous and post even more exaggerated accounts of their own lives. In the end, on Facebook, everyone else looks like they’re doing better than you. The downside of this focus on pre-existing news is that it’s impossible to demonstrate causality – we don’t know that viewing content on Facebook actually causes you to think or behave differently. To demonstrate causation (not just correlation), you need an experiment, and that’s exactly what these researchers did. The result? Manipulating viewed post mood did affect the apparently mood in posts made by the study participants – but the effect was tiny (d = 0.02 was the largest observed effect). So it makes a difference – but perhaps not as big of one as previous research has suggested.

Reactions have been mixed. Psychologists and science writers have tended to focus on explaining the study to non-psychologists, whereas popular sources have focused (as usual) on the outrage.

Facebook-controversy-related comic from cad-comic.com

Facebook-controversy-related comic from cad-comic.com

With that, we come to the heart of my title-question: Is the outrage over the Facebook mood manipulation study mere ignorance or anti-science? Let me explain with three points.

First, we know that web media companies, especially those who have hired marketing consultants, engage in internal experimental research with our data and our moods all the time. They do this to improve the attractiveness of their products. By randomly assigning people viewing web content to two different experiences, they can observe the difference between the two in terms of time spent on the site, information shared, and a variety of other metrics. Data in hand, web developers can then change their entire site to the format that was most successful in experimental testing. Websites like Amazon, Yelp, and Facebook all do this with user-contributed data, altering the algorithms that shape what you see. Even Wikipedia’s example of this approach involves randomly assigning people to receive different marketing emails. For people that didn’t know this approach was so common, despite agreeing to Facebook’s terms of service that indicate your permission for them to experiment on you freely, their outrage is likely driven by ignorance. I suggest these people go delete their Facebook accounts right now, or at least read the terms of service more carefully.

Second, if you are aware that Facebook is collecting your data and manipulating you, but you didn’t find the Facebook experience to be worth that loss of data control, you should have also deleted your account by now. As Facebook is a private company with a product they are trying to sell you, you are perfectly free to give up on that product at will. You can’t be mood-manipulated if you don’t use their service. If you’re angry and haven’t deleted your account, you have already, through your actions, demonstrated that access to Facebook is more important to you than avoiding being manipulated by a massive technology company.

Third, if you already knew that Facebook was collecting your data and manipulating you, the major difference with this study and the hundreds of studies that Facebook has undoubtedly already done is that it is published in the scientific research literature. Most research like this goes on behind closed doors. You know that Big Data is processing all of the status updates that you provide to Facebook and that experimental manipulations are being used to compel you to stare at Facebook longer. But this time, they went public with the results – and only now is there anger.

From xkcd.com

From xkcd.com

Having said all that, could the researchers have handled this better to minimize negative public reactions? Certainly yes.

First, obtaining informed consent from participants that an experiment was going on would undoubtedly have biased their results, so obtaining consent ahead of time would not have been possible – a justifiable risk, given the exceptionally low probability of harm (in the terms of IRB, “minimal risk”) to participants in this study. However, it would have been courteous to have either debriefed participants (even now, we don’t know exactly who was manipulated) or given participants the opportunity to withhold their data from the final dataset, even if users have explicitly agreed to be experimented on in their day-to-day interactions on Facebook.

Second, the ethical review process for this study was business-as-usual for many IRBs – in other words, orchestrated by the researchers to minimize the hassle they needed to go through to meet the minimum possible institutional requirements for that review.  Facebook apparently conducted an internal review of the study protocol proposed by the first author (a Facebook employee), presumably in consultation with his two Cornell faculty co-authors. No surprises there – Facebook manipulates its users all the time, this manipulation was within its terms of service, so why would they care? Once the data were collected, the Cornell faculty authors applied to their own IRB to use the data collected by Facebook as “pre-existing data.” Given the sheer scope of this study, knowing that they’d be trying to alter the interactions of three-quarters of a million people, the faculty would have been better off sending the study for full IRB review.  Having said that, a full IRB probably would have approved the protocol anyway because the manipulation was well in line with what Facebook already does on a regular basis anyway – the researchers just redirected Facebook’s business-as-usual to conduct science instead of, or perhaps in addition to, profit.

Footnotes:
  1. Kramer, A., Guillory, J., & Hancock, J. (2014). Experimental evidence of massive-scale emotional contagion through social networks Proceedings of the National Academy of Sciences, 111 (24), 8788-8790 DOI: 10.1073/pnas.1320040111 []
Share

SIOP 2014: Reflections on SIOP Honolulu

2014 June 19
tags: ,
by Richard N. Landers

SIOP 2014 Coverage: Schedule Planning | Live-Blog | Reflections

Normally, SIOP is held during the final weeks of April. It’s probably the worst conceivable timing for the academic community – immediately before and sometimes during finals of the Spring semester. This year, SIOP was unusually scheduled due to its unusual location: Honolulu, Hawaii. With the wise assumption that many SIOP attendees would dual-purpose their trip to Honolulu into a vacation, the conference was pushed forward to mid-May (and post-semester). The daily conference schedule was also pushed forward, starting at 7AM and ending around 3PM, in order to allow people to enjoy their afternoons in Hawaii. That resulted in a very different SIOP feel than usual. Not really better or worse – just different.

My personal conference schedule was packed full, with 8 separate events (9 total presentations) spread over all three days of the conference. That didn’t leave a whole lot of time to attend other people’s sessions, to listen and reflect, which is really the best part of SIOP for me. My overall goals in attending SIOP are to be inspired in my own research, to incorporate new ideas into my thinking, and to remind myself of the excitement of research and why I took a research career in the first place. Even with my schedule, I think these goals were met.

In terms of specific content, I also use SIOP as a barometer of the tech sophistication of the practitioner HR consultant crowd. I used to think the Academy of Management conference would be the best place for this, since they do study management after all, but after attending AOM for a few years, I realized that the Academy crowd is not generally very concerned with people are actually doing in real organizations. So now I stick with SIOP to learn what I need to know.

In general, things are looking good for technology in IO. At a minimum, people are paying attention and heeding the calls I’ve been making for years – namely that technology is becoming an overwhelming force of marketing in the HR consulting community, with both good and bad implications, and we need to pay attention.

On the good side, technology enables us to do a lot we couldn’t do before. One of the most fascinating examples I saw was a manufacturing simulation in which consultants could simulate an entire car-building process, recording every mouse click, every physical movement, every eye glance of the operators for the purpose of either developmental or administrative assessment. For softer skills, assessment is improving but isn’t yet great. Leadership assessment, for example, usually takes place online and is video-based, in the tradition of situational judgment tests (SJTs). That’s more convenient than an in-person role-play, but we can do better. The line between SJTs and assessment centers is blurring, and I think this will ultimately force purveyors of both to really think about whether they are really unique concepts and what unique value each brings, if any.

On the bad side, technology is edging out the more important part of the equation – the underlying psychological mechanisms that enable these technologies to work in the first place. In many cases, assessment technology is more about technology than assessment; learning technology is more about technology than learning. That is not the way things should be, and the IO crowd is starting to take notice. We are the ones to make that stand, and we need more people fighting the good fight!

A great example of this is the recent highly visible gamification effort by Knack called Wasabi Waiter. It is essentially a casual web browser game (reminds me of classic casual game Diner Dash), but it purportedly can be used to select high performers on “social intelligence” among other constructs. The media has gone crazy over this thing. Yet there is no published psychometric evidence or even a white paper describing the construct validity of whatever it is that this game is supposed to assess. Without that, we should not trust this game – it may be all flash and no substance – yet the press is impressed with flashy, and that’s what people remember. IO should bring the science!

Another assessment gamification effort called Insanely Driven provides another case study that I learned about at SIOP. This game supposedly assesses candidates on… something. I honestly don’t know. And I’m not convinced the creators do either! It’s an entertaining experience, and personality profiles are spit out at the end, depending upon your performance, but is it valid? Again – nothing published, nothing shared, so who knows?

In any case, next year’s SIOP is not anywhere quite so exciting (Philadelphia!), so we’ll see then if the tech enthusiasm gets even stronger!

Share
LinkedIn Auto Publish Powered By : XYZScripts.com