Wikiversity:Community Review/CR process discussion

Proposed changes to community review
This page is an experimental Community Review about Community Review. The experiment is pretty simple, and is based on methods borrowed from Wikipedia's ARBCOM.

The rules to this experiment are simple: rather than using threaded discussions, we'll have separate sections for each participant to make their statements in. Each participant may only add comments to their own section, and each person's section must have at most 700 words. Comments on other people's comments need to be made in the person's own section as well (try phrasing with "@this_guy, @that_guy, etc.", but within the 700 word limit including both statements and responses.

Statements can and should be edited and refined over the course of the discussion, so that the person who will eventually be in charge of closing the discussion will read the edited and refined versions rather than having to wade through everything from the initial gut feeling all the way to the final opinion (which may or may not be different).

Note that this introduction and explanation took less than 200 words.

1. Should all community reviews be run like this one?

2. Should there be a limit on the number of Community Reviews opened by any particular person during some particular time period?

3. Should we continue to transclude CRs from the main page, or just link to them?

4. Should community reviews require a "second witness" before they are initiated?

5. Should all current reviews be listed in sitenotice, Colloquium etc.?

6. Should reviews which are inactive for a period of time be closed?


 * Note: This page was copied from userspace. --mikeu talk 16:06, 9 February 2011 (UTC)
 * Note: Concrete resolutions related to a number of the topics discussed here are being considered for adoption at Community Review/CR process discussion/Resolutions. --SB_Johnny talk 14:52, 12 February 2011 (UTC)

Comments from Moulton
Wikiversity definitely needs a more dignified and functional process for feedback, self-governance, and dispute resolution.

It will come as no surprise to anyone that I favor a Social Contract Community Governance Model, with a Community Agreement that includes a Code of Scholarly Ethics and a functional Dispute Resolution Process.

@SBJ: I suppose my besmirched reputation precedes me.

@Mike: In Robert's Rules of Order, a Motion needs at least one "Second" to be placed on the agenda. Perhaps we need a requirement that a CR cannot be filed unless at least two or three people jointly file it.

@Abd: Your comments add up to over 1300 words. Thank you for not abusing the participants here with more verbiage than they have time, interest, or patience to attend to.

@SBJ: Regarding Jerques and Clerques — I'd prefer a solution based on a sincere commitment to higher ethical standards rather than one base on more bureaucracy.

Moulton

Comments from SB_Johnny
I'm opening this up to see if there's a better way of making CR's meaningful, useful, and satisfactory. CR seems badly broken right now, mostly because they've been used lately for personal agendas. I'd like to see them become a community resource rather than a "weapon".

Rejigging to follow the "response to each question" approach pioneered by Abd and Jtniell (and which I think is a good way forward):

1. This first point should probably be broken up for comments:

1a. Should community reviews use a non-threaded approach like it's been done on this page?
 * I'd say yes, as it seems to allow for concise statements.

1b. Should we try to organize the statements as short paragraphs to comment on each point brought up?
 * Again, I'd say yes, as it seems to allow for concise statements. The end goal should be to have a sense of the community's opinion, which would be far easier to divine when they're organized in this way.

1c. Assuming 1a and 1b, should we try to keep comments to 700 words (or an arbitrary limit along those lines)?
 * Again, I'd say yes, as it seems to allow for concise statements. We can always use the talk page for unabridged debate.

1d. Should additional questions for review be added when one participant raises it, and another thinks it's worthwhile to move to the top?
 * We've sort of organically fallen into that, and it seems a good practice, similar to "seconding".

1e. Should there be a limit on the number of questions considered during a particular community review?
 * I'd say yes, maybe 5 or 6. If there are more than that it becomes too non-specific, or perhaps worth more than one review.

2. I think #2 is probably unnecessary, if we adopt #4. However, see me comment on #3.

3. I think it's OK to transclude them, as long as we're not running too many of them at once. Perhaps we could transclude up to two of them (the two most active ones, maybe?), but more than that would be a bit much.

4. Yes.

5. On the Colloquium yes, but only as an announcement (as opposed to a parallel debate).

6. Not sure yet.

Actually, I think if we're going to move this to a regular CR, maybe it would be best to focus only on the structural part (question 1 and the "subquestions", and questions 2 and 4). It would be a lot easier to handle the other questions in a second CR after defining the basic structure.

@Jtniell on #6 and #4: maybe what's needed is a "clerk"?

Comments from LauraHale
I'm cool with non-threaded. I like the idea of doing something like Fan History's multiple perspectives and meatball wiki which basically allow users to insert their own, non-disrupted thoughts that are then later integrated into a more cohesive narrative. This actually feels more wiki like to me than threaded conversations. --LauraHale 22:48, 2 February 2011 (UTC)

Answer from Ottava
To number 3 "Should we continue to transclude CRs from the main page, or just link to them? " - Meta links them all by month filed. Ottava Rima (talk) 01:01, 3 February 2011 (UTC)

Proposal by Ottava Rima
Ban all use of terms derived from Robert's Rules of Order that aren't used by Wikiversity or the WMF projects. All of these statements about point of order, tabling, etc, detract from what a discussion is. Consensus is a compromised agreement based on satisfying the concerns of people. It is not a black and white decision and using pseudo-legal jargon takes away from that. Ottava Rima (talk) 21:04, 8 February 2011 (UTC)

Comments from Mu301
Re: #1 I'd say we should move this to WV:CR and give this process a try before deciding to use in other reviews. Overall, I think that we desperately need some change to the current methods of conducting CR, though I'm not sure what specific changes would be of most benefit. Re: #2 I would say so. If there really is an outstanding need for multiple reviews they can, of course, be opened by another member of the community. Having too many going at once with overlapping scope just confuses the issues. Re: #3 The transclude seems to be a good way to do things, though we do need to pay more attention to the level of section headers used. There are also some ideas here that we might want to consider. I would also like to see future reviews start with a discussion of the perceived problem, instead of a single solution being proposed when the review is created. Several of the recent reviews jumped too quickly into voting, imho. --mikeu talk 15:45, 3 February 2011 (UTC)
 * @Moulton: The idea that a review needs a "second" to proceed would improve the review process, imo. --mikeu talk 16:17, 3 February 2011 (UTC)

Comments from Abd
The original version of my comment is at . I intend to refactor this, to condense it, as the rest of the discussion here proceeds, so that it's my mature -- and far more concise -- comment. For now, consider it "preliminary discussion." --Abd 22:30, 3 February 2011 (UTC)

Some additional process comments and banter have been removed, see.

@SB_Johnny: @Abd: I'm sure you can say it in 700 words or less. Indeed I can. "It." Or, in a variation on my reply in Talk: "Bullsh it!" ---Abd 23:39, 3 February 2011 (UTC)

@Moulton. Really? Moi? Ah, you looked at my draft! Hey, if you don't like my comments, think they are too long, please don't read them! I agree with limits on the final version of each section, but not to any limit on what can be incorporated by reference. --Abd 23:39, 3 February 2011 (UTC)

The 700 word limit is less than half of what ArbComm allows in the filing and evidence pages, with no limit on the Workshop sections. (The Workshop is where the train wrecks mostly take place. Threaded discussion.) --Abd 23:43, 3 February 2011 (UTC)

5. Should all current reviews be listed in sitenotice, Colloquium etc.? No. Any user may raise any question, normally, at the Colloquium. However, only reviews that are approved as worthy of community attention, by an administrator, should be listed in the site notice. The issue of listing can be discussed on Talk for the Review (best) with a pointer from the Colloquium, or at Request custodian action, since a custodian action is required. (No involved custodian should list his or her own filing.)

6. Should reviews which are inactive for a period of time be closed? Closure for inactivity doesn't address concerns. However, where consensus is clear, individual review sections may be and often are closed. Other sections theoretically remain open. No specific time limit should be set. Anyone may close any discussion, including the whole thing, without a consensus, it's an ordinary edit, and if someone wants to reopen that, there are two ways to do so: revert the close or add a new section or discuss it on the attached Talk page. --Abd 16:48, 4 February 2011 (UTC)

Process defect
7 resolutions were developed by SB_Johnny and proposed with, without the agreement of the virtual committee discussing this CR, and this was placed in the site notice promptly. The proposed changes limit the number of resolutions to be presented to 7, therefore this was practically a close of the CR, proceeding to !vote, without discussion of the exact form of the questions, which is crucial. I'm not criticizing SBJ's attempt to crystallize consensus here, but CR must become a formal process, with process safeguards against domination by any individual. Including me, by the way. As proposed resolutions, fine. But the resolutions page was then listed in the site notice, without clear reference to the underlying CR. That's backwards! People !voting should have at least a glance at the CR itself before voting on resolutions coming out of it!


 * Careful deliberative process has rigorous protection for minority opinion, without allowing small minorities to disrupt the process. We should learn from it. It can take extra time, on-line, but the resulting increased consensus is well worth it. --Abd 16:40, 13 February 2011 (UTC)

Comments from Guido den Broeder

 * 1) Absolutely. I think a total of 700 words is still a lot and should make nobody uncomfortable. I, for one, don't expect ever needing more. Guido den Broeder 23:35, 8 February 2011 (UTC)
 * 2) A review where opinions may depend on the outcome of an already ongoing review should wait until the latter is closed. I have no objections to the opening of two or more unrelated reviews. Guido den Broeder 23:40, 8 February 2011 (UTC)
 * 3) Transclusions are confusing, especially to new users. They serve little purpose, so let's get rid of them. Guido den Broeder 23:43, 8 February 2011 (UTC)
 * 4) No. A second witness is very easy to produce and a possible lack of sincerity or the existence of ulterior motives may cloud the original issue. Guido den Broeder 23:48, 8 February 2011 (UTC)
 * 5) The colloquium. Also, involved parties should receive notice. Guido den Broeder 23:50, 8 February 2011 (UTC)
 * 6) Rather, there should be a maximum duration. We'd want to avoid a situation where someone purposely keeps a review open by editing the page every x days. Guido den Broeder 23:54, 8 February 2011 (UTC)

Comments from Jtneill
-- Jtneill - Talk - c 16:02, 4 February 2011 (UTC) -- Jtneill - Talk - c 13:47, 5 February 2011 (UTC)
 * Proposed changes to community review
 * 1) Should all community reviews should be run like this one?
 * 2) * Partly. Some aspects could be helpful, particularly requesting summative comments from contributors. But its harder to follow dialogue in the way that threaded discussion allows. So maybe have two sections in a CR - threaded discussions (could be on the talk page) followed by a more concise section for contributors' summative comments. I guess it's kind of like this already where there is discussion and then a proposal with summative comments and vote per user.
 * 3) Should there be a limit on the number of Community Reviews opened by any particular person during some particular time period?
 * 4) * No. It could be one way to encourage people to approach starting CRs more carefully and cautiously. However it may lead to even more large and unwieldy CRs because someone may seek to address multiple issues in a single review. I actually think smaller and more focused reviews have a greater chance of being more quickly and satisfactorily resolved, so would be hesitant to put a cap on the number of reviews initiated by users.
 * 5) Should we continue to transclude CRs from the main page, or just link to them?
 * 6) * Remove transclusion. Time's up on transluding - we need a cleaner WV:CR page which lists and links to current reviews.
 * 7) Should community reviews require a "second witness" before they are initiated?
 * 8) * Maybe. I think some more checks and balances are needed before initiating a review. Rather than a "second witness" perhaps we focus on making explicit minimum criteria for a CR - and if the criteria has not been met, the review is moved to a sub-page of the nominator.
 * 9) Should all current reviews be listed in sitenotice, Colloquium etc.?
 * 10) * The community should be informed. They generally were until recently, but the recent increase has made people hesitant to bombard sitenotice and to some extent Colloquium with constant CR reminders. If more checks and balances are applied, then perhaps CRs can continue to be announced. Perhaps a template providing a list of current CRs could be used to to help alert people about current CRs.
 * 11) Should reviews which are inactive for a period of time be closed?
 * 12) * Better monitoring needed. Its relatively easy to open a review; and much less easy to find mutually satisfying conclusions. Current reviews consist of several relatively dormant ones and a few which are active. Perhaps reviews can be listed on WV:CR in a sortable timetable with date of filing and most recent activity to give a dashboard indication of CR status.