8
$\begingroup$

One of the big problems we are having is the number of poorly referenced answers on the site. They often make unsubstantiated and contentious claims without any supporting material, and that's not the standard we should be accepting. Until recently, we had three options on what to do with an uncited answer/statement:

i) Comment and request references, ideally drawing attention to specific points of contention

ii) Downvote (though an accompanying comment would be useful)

iii) Add references yourself to other peoples answers if you think you can provide the material needed

A recent feature was added whereby we could flag for moderator attention and a banner would be added below the answer.

Some of the information contained in this post requires additional references. Please edit to add citations to reliable sources that support the assertions made here. Unsourced material may be disputed or deleted.

However, this has been largely ineffectual.* Most of these banners, along with comments and downvotes, are ignored by the original poster and answers continue to attract upvotes. If we continue to allow poor standards of content checking we'd be no better than PLoS One with the Creator's hand paper (or #AddMaleAuthorgate)...

I suggest we need to find a more effective way of enforcing the provision of citations and references where required.

I've posted some possible answers, but by no means do I think one or any are the perfect solution, and this should be a discussion - answer, comment, and edit posts as necessary (e.g. to add pro's and con's). This is intended to be an open discussion, and abstract and creative solutions are very welcome! Any weaknesses in solutions can be much more easily dealt with as a collective group.

Also it's key to discuss how much time is it fair to give a user to support their material with references before we were to start punishment?

* Examples: Answer 1, Answer 2, Answer 3, Answer 4, Answer 5, many answers here

$\endgroup$
11
  • 1
    $\begingroup$ There is no provision for implementing the first two suggestions. You are essentially asking for a feature-request which only a developer can implement. Moreover such features would be implemented only if they are generally useful to most of the sites in the SE network. $\endgroup$
    – WYSIWYG
    Mar 4, 2016 at 10:09
  • 1
    $\begingroup$ I was hoping for a discussion of what sort of feature would be suitable before making a feature request. E.g. I would make a feature request to "allow reputation penalties as a measure to improve referencing", rather than making a feature request to SE that is just "please make people reference more" $\endgroup$
    – rg255
    Mar 4, 2016 at 10:14
  • 2
    $\begingroup$ @rg255 If I had known my single slip in not referencing an answer would have such grave consequences, I would honestly never, ever answered that question... Forgive me, for I have sinned. $\endgroup$
    – AliceD Mod
    Mar 4, 2016 at 10:27
  • 1
    $\begingroup$ haha, I loved that response to a reference request, slight overkill!! $\endgroup$
    – rg255
    Mar 4, 2016 at 10:29
  • 1
    $\begingroup$ Another problem with adding a post notice is that a lot of unregistered new users write such answers which are only on the site once and never come back. $\endgroup$
    – Chris Mod
    Mar 4, 2016 at 11:06
  • $\begingroup$ Some form of proactive measure would be good - comments, downvotes, and banners and other reactive measures (rep fines, hiding the answers) will never work for those. $\endgroup$
    – rg255
    Mar 4, 2016 at 11:32
  • 1
    $\begingroup$ I think an effective method would be to lock upvoting and allowing only downvoting on unreferenced or poorly referenced answers till proper references are added. Also, for what @Chris said, we can disallow answering by unregistered users. $\endgroup$
    – another 'Homo sapien'
    Mar 11, 2016 at 10:37
  • 1
    $\begingroup$ Is there no flag for removal due to lack of referencing? Surely that would be problem solved. $\endgroup$
    – James
    Mar 14, 2016 at 3:40
  • $\begingroup$ Currently a flag for lack of references results in the banner (as described above). Mod deletion via flags is for "serious" problems (when posts don't answer questions etc.). The other route for deletion is by high rep users (>20k, which we only have a very small number of). We could start a meta thread for answers we want deleting/need to discuss on the basis of lacking references there, this would highlight such posts for the >20k users, effectively giving the <20k users some input on this. $\endgroup$
    – rg255
    Mar 14, 2016 at 8:34
  • $\begingroup$ @another'Homosapien' would you like to add those as answers (i) vote locking and ii) disallowing new users)? $\endgroup$
    – rg255
    Mar 14, 2016 at 9:36
  • $\begingroup$ I don't know if you've noticed the irony in your suggestion but the PloS One scandal (and others like it) happened in the very journals that you're suggesting we should be more strict about citing in our answers. I completely agree with the proposal that citations should be encouraged, but care still needs to be taken to ensure it's not and end in its own right. Avoiding ending up like PLoS One, Sokal etc is about more logical scrutiny, not more citation. $\endgroup$
    – Isaacson
    May 16, 2017 at 6:26

5 Answers 5

Reset to default
3
$\begingroup$

Deletion

Deleting answers which do not meet the referencing standards we strive for.

How it could work

High rep users can vote to delete the post on the basis of lacking (suitable) references. After 3-5 votes the post is deleted.

Pros

  • Existing system for votes to delete
  • Community democracy
  • Effective mechanism against regular users and one timers
  • Can be undeleted (?) so is reversible

Cons

  • OP would need to be given fair time and opportunity to make the requested edits, making it a slow response, allowing poor posts to persist for an unnecessarily long time in the public domain
  • Could remove content which has the potential to be good for the site (some answers just need some references to make them good - "diamond in the rough") so we would need to be cautious about what gets deleted
$\endgroup$
1
  • 1
    $\begingroup$ I think deletion is my preference from what is currently listed, so it's a virtual +1 from me! $\endgroup$
    – rg255
    Mar 14, 2016 at 11:54
2
$\begingroup$

Short answer- I think an effective method to solve this problem would be to lock upvoting and allowing only downvoting on unreferenced or poorly referenced answers till proper references are added (just like that line 'some parts of this answer require proper references...' is shown below such answers). Also, as @Chris said in comments, we can disallow answering by unregistered users.

Expanded by rg255:

Locking out (up)votes

Answers without suitable references can be locked from being (further) upvoted.

How it could work

Once the post has been flagged, and the citations needed banner added, we could simultaneously block upvoting on the answer. This would prevent further rep gain from answers until suitable references are added. Moreover, we could also stop -1 reputation fine on downvoting the answer. Many users avoid downvoting by the fear of getting -1, this could be prevented on answers lacking citations.

Pros

  • Prevents gain of rep from poorly referenced answers
  • Allows downvoting and feedback from the entire community (incl. low rep users)
  • Can be reversed

Cons

  • The post is still part of the wider community, and may be read (and treated) like a quality answer by visitors
  • No existing SE system
$\endgroup$
2
  • $\begingroup$ Locking affects all votes and comments and can be performed only by the mods. Locking, moreover comes with non-custom fixed "reasons" such as content dispute, off-topic comments, historical significance and wiki-answer, which are generally not compatible with the reason intended above. Locking for a historical significance is the one that comes closest but it is applicable only for questions. $\endgroup$
    – WYSIWYG
    Mar 15, 2016 at 9:03
  • 1
    $\begingroup$ @WYSIWYG I agree with you. I think 'content dispute' might be a considerable choice because the problem is in the content: it does not have a proper reference (I actually don't know what 'content dispute' stands for when locking votes). $\endgroup$
    – another 'Homo sapien'
    Mar 15, 2016 at 10:25
0
$\begingroup$

Reference requirement input

When typing an answer, have a required input field for references.

How it could work

When a user types a response there could be another input box that docks several points if left blank (of course they could be regained if the user adds references). When clicking the post your answer button, if the box is blank display a notification.

When the answer is posted the references would be on the bottom. Preferably links but MLA could work.

Pros

  • Offenders would be notified of their offence before posting
  • Any offense could easily be corrected
  • Will remind people to add references (is proactive)
  • Not overly excessive

Cons

  • May require some programming
  • The SE system may not allow for it
  • It could be to light of a punishment system for noncompliant users
  • Possibly easy to work around (just putting anything in the obligatory box to trick the system)
$\endgroup$
3
  • $\begingroup$ What does MLA mean? $\endgroup$
    – rg255
    Mar 13, 2016 at 8:24
  • $\begingroup$ @rg255 mla is a citation standrand its probably the most commonly used. $\endgroup$
    – tox123
    Mar 13, 2016 at 14:47
  • 1
    $\begingroup$ I think this is good because it's proactive, but could be easy to work around by simply posting anything in the obligatory field... could also achieve the same by having an automatic pop-up (reminding to give references) which comes up if no links are in the main text when the OP tries to post the answer (like when you send an email without attachements and in the email text it says "I've attached the file", Gmail notifies you before sending to check) $\endgroup$
    – rg255
    Mar 14, 2016 at 9:04
-1
$\begingroup$

Reputation Fines

Removal of (large) chunks of reputation points from users for failing to give references.

How it could work

High rep users could flag answers as not having (good/suitable) references, ideally giving detail on the sort of references needed. Mods will see the raised flags and judge the need for references. The current banner regarding lacking references will be added to the answer and the user notified (via inbox message). It will also carry a warning on the post that in $X$ days they will be docked $Y$ reputation points unless references are added.

I'd suggest that one or two weeks is a reasonable amount of time. Rep docking of all the points accrued by the answer + a 10% penalty (e.g. an answer has 5 upvotes, giving it a value of 50 rep points, the user is penalised 55 points) would, in my opinion, be a fair penalty.

If the OP (or another user) does add references they can flag it for mod attention to have the points reinstated, or it can go in to an automatic reimbursement queue. If the OP disagrees with the fine (thinks references do not need to be added), they can flag to appeal the penalty, and if the appeal is granted the points are reinstated.

Pros

  • A strong measure
  • Takes away the benefit of posting the answer (rep points)
  • Can be reversed (the OP has the opportunity to make the requested edits, and the opportunity to earn their points back)
  • Allows posts to remain indefinitely, allowing them to be saved/improved in the future
  • Could be very effective against serial offenders (there are several such users) who have a lot of rep to lose, and are more personally invested in the site (i.e. more likely to be bothered by their disappearing rep)
  • Posts can be affected quickly

Cons

  • No existing SE mechanism in place
  • Requires mod intervention, making it less democratic and more work for the mods
  • It may be an overly aggressive strategy
  • Quite ineffectual against one time or very infrequent users
$\endgroup$
3
  • $\begingroup$ You don't seem to have any references here. $\endgroup$
    – tox123
    Mar 13, 2016 at 22:24
  • $\begingroup$ Another problem is that it wouldn't affect new users. Eg. If a user has rep. 1 and gets 3 upvotes in answer without citation, then by reducing their rep. also the user will have same rep. as earlier i.e. it will have no negative effect on them. $\endgroup$
    – another 'Homo sapien'
    Mar 14, 2016 at 10:59
  • 1
    $\begingroup$ They would lose the rep gained from the answer, but obviously wouldn't be able to go below 1 rep so the 10% not be taken if it is their first answer. Following your example the new user would have 16 rep (1 for joining, 15 for 3 upvotes) they would lose the 15. $\endgroup$
    – rg255
    Mar 14, 2016 at 11:03
-3
$\begingroup$

Hidden Answers

Answer posts are hidden from view to users unless they have a high level rep. This would allow trusted users to give feedback on the post, while preventing new users etc. from seeing it. This would reduce the number of upvotes given, and help keep posts that do not meet standards from being visible (thus not allowing them to serve as examples of acceptable posts).

How it could work

High rep users directly vote to hide the post, on the basis of it needing references. The post is hidden when 3-5(?) votes are registered, and the OP is requested to add references. Once references are added the post is automatically added to a queue to vote on making it visible.

Pros

  • Helps maintain site standards seen by new users
  • Allows the OP to improve the post
  • Prevents the OP from gaining (significant) rep unless they make the requested improvements
  • Allows the post to remain on the site indefinitely while it waits for improvement, meaning the post can be rescued in the future (it's reversible)
  • Democratically controlled by community rather than mods
  • Posts can be affected quickly (the community can impose this immediately upon seeing lacking references)

Cons

  • No existing SE system in place
$\endgroup$
1
  • $\begingroup$ There's a flaw in your logic here, you first suggest that poor quality answers could potentially get high votes "This would reduce the number of upvotes given", then suggest that high rep users can be relied upon to provide quality answers. They can't both be true. If there's a problem with lower quality answers getting high votes, then there must also be some low quality answerers who have high reps as a result. Contrarily if high rep users are there because they consistently provide quality answers, then the voting system must be working and the low quality answers will be shunted down. $\endgroup$
    – Isaacson
    May 16, 2017 at 6:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .