Apologies if this is answered elsewhere. I looked through other posts but I don't think my question is answered.  I have a situation where 2 coders rate whether a series of research papers contain particular elements. The issue is that they can choose any number of elements (we offer n possible elements to choose from, but they simply code if they think each category is present and it is up to them how many categories are present in total).  So coder 1 might indicate that 1, 4, 6 and 10 are present. While coder 2 might indicate that 1, 3, 6, 9, 10 and 15 are present.  I am not sure how to appropriately analyse the rates of agreement.  OPTION A (which we have done, I think incorrectly) is to have a row for each possible element in each paper (so paper 1 element 1, paper 1 element 2,… paper 1 element n, AND THEN paper 2 element 1, paper 2 element 2 etc). Then we have columns for the raters (multiple raters but only 2 raters work on a particular subset of papers) see   We were then using this syntax in SPSS to try to calculate kappa (question is not about SPSS but in case it provides useful context) CROSSTABS   /TABLES=Coder_12 BY Coder_13   /FORMAT=AVALUE TABLES   /STATISTICS=KAPPA   /CELLS=COUNT ROW COLUMN   /COUNT ROUND CELL.  However, for complete agreement this would require the coders to have the same number of elements chosen as well as the same specific elements chosen. I am not sure this is the best approach? It gives us one kappa for the pair of coders for all elements in all papers they both coded.   OPTION B would be to just to compare (for each paper) whether the 2 coders agreed element 1 was present or absent, and then whether element 2 was present or absent etc. This would result in a large number of kappa’s for each pair of coders (the total number of papers multiplied by the total number of possible elements).  I am somewhat unsure about this option given the number of kappa’s it generates.  OPTION C - something else that I am not aware of 😃  any advice would be greatly appreciated.
I have over 400 open-ended responses. The total has been coded by one rater, and a 10% identical sample of the total has been coded by 2 raters. There is a choice of 14 nominal codes, of which the rater can use any number per case, though most cases have between 1-4 codes. Which statistical test is the most appropriate to determine if I have inter-rater agreement on the 10% sample?   Also am I right in assuming that I will need to treat each code as a variable and look for the level of agreement for each variable?   I'm using SPSS 21.  Much thanks, Kellie
Error installing Apache2 Server Wordpress speed slow after .5 million single image post via wp-cli What prevents this magnetic perpetuum mobile from working? Why are Alkaline, NiMH, and Li-ion AA measured differently (mAh, mWh, etc)? Looking around Amazon, it seems that different types of batteries are measured differently:  Does anyone know why?   Li-ion are measured in mWh    (Cheap-Os)  3000mWh AmpTorrent  NiMH are measured in mAh      Alkalines are typically not measured   "Long Lasting" Duracell "Longest Lasting" Energizer    I assume that Alkalines are not measured for some sort of historical reason with marketing departments?  ... or is it that alkalines so mystical and unreliable that giving them a measurement is too aggressive? Or that switching to non-toxic / non-mercury was such a bad move in performance that measurements had to be removed in the 90s so people wouldn't know?  Perhaps NiMH are measured in mAh because the voltage can very so greatly and mWh doesn't make sense?  Are Li-ion measured in mWh because that's more consistent with non-battery measurements and they do deliver reliable voltage? New global search doesn't like tags I am trying to install Ubuntu 14.04 on FakeRAID along Win7.  Win 7 has been already installed, and dmraid is running.  During the installation I choose to manually partition, this is the structure:  \dev\mapper\isw_gchdghhd_Volume0 \dev\mapper\isw_gchdghhd_Volume0Op1 - NTFS Win7 loader \dev\mapper\isw_gchdghhd_Volume0Op2 - NTFS Win7 partition \dev\mapper\isw_gchdghhd_Volume0Op3 - swap \dev\mapper\isw_gchdghhd_Volume0Op5 - ext4 mounted as \ \dev\mapper\isw_gchdghhd_Volume0Op6 - ext4 mounted as \home   There is another sequence similar to above marked as "linear".  I choose \dev\mapper\isw_gchdghhd_Volume0 as boot device.  When immediately after the installation begins, an error with question marks pops up, and the installation stops. How to take the gradient of the quadratic form? It's stated that the gradient of:  $$\frac{1}{2}x^TAx - b^Tx +c$$  is   $$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$  How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$? List creation/manipulation I have a package (fcl) in my PPA  in my launchpad PPA and it depends on another package (libccd) which is in another PPA ppa:imnmfotmal/libccd. fcl has both build and run dependency in libccd. So i added libccd as a PPA dependency in launchpad and both are successfully build. but when i am trying to install fcl package after adding the fcl PPA, its throwing an error Reading package lists... Done Building dependency tree        Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation:  The following packages have unmet dependencies:  fcl : Depends: libccd but it is not installable E: Unable to correct problems, you have held broken packages.  but its successfully getting installed after adding the libccd PPA also. I want fcl to install libccd automatically without adding its PPA. how can i achieve it ?? Output of apt-cache policy fcl fcl:   Installed: (none)   Candidate: 1.0-0ppa5   Version table:      1.0-0ppa5 0         500 http://ppa.launchpad.net/imnmfotmal/fcl/ubuntu/ trusty/main amd64 Packages The workings of the Hall effect? I want to ask about the workings of the . Why do the electrons come to rest on the edge of the wire? The magnetic field pushes them up, and the electric field pushes them forward. Shouldn't they reach the top of of the conductor and then stop moving up but continue moving forward?  Why does the magnetic field stop them from continuing to flow? The magnetic force is up so why would it effect their velocity forwards? I can't play online. I can only play solo. I have bought Minecraft so it should work fine, but, when I try to connect to a server, it says:  Internal client error: java.net.UnknownHostException: session.minecraft.net   I have tried over 20 different servers and it says the same thing for every single one.   Note: These are not Hamachi servers. I'm attempting a direct connection. Would you be weightless at the center of the Earth? If you could travel to the center of the Earth (or any planet), would you be weightless there? How to make terminal autocomplete when there are several files/directory? If I have several directories, like:  afoo abar   sometimes my terminal will refuse autocomplete when I press tab (e.g. "cd a" then tab), and print the list of directories instead. Sometimes it even throws a noisy, annoying sound. Any idea how to make it autocomplete in cases like this? E.g it can show abar first, and then afoo if I press tab again. I saw this is the case in windows, or some applciation in Ubuntu Cannot do a `composer install` even though my minimum stability is listed as dev I am trying to build the master branch in my project after updating to 8.5.0, but am being met with an error from composer (which I did not get on my develop branch):     The requested package drupal/focal_point 1.0.0-beta6 is satisfiable by   drupal/focal_point[1.0.0-beta6] but these conflict with your   requirements or minimum-stability.   In my composer file, I have:  "drupal/focal_point": "^1.0",   as well as:  "minimum-stability": "dev",   How can I fix this so TravisCI gets unstuck? It worked fine for the develop branch, and the master is now updated with it. close votes vs votes remaining I am new to the review part of Stack Overflow and I am trying to understand what is the meaning of votes remaining in the Review | Close Vote.  For example: Today I was at my 39 vote and it said that there was still 21 votes remaining.        However, after my 40 vote:        So, apparently I have started with 60 votes but I can use only 40? How do they film continuous blood pooling? Load R package from character string My employer didn't send me W-2 for December 2015. In fact, he told me that I wasn't on payroll for the two months(Dec/Jan) I worked for them so I am not sure whether he will be able to generate W-2 for me or not. I am wondering should I call IRS and inform them about the same, and probably they will call the employer? Or should I report my December 2015 earnings as Miscellaneous and report to IRS while filing taxes? Or what other options do I have? I am a non resident alien on F-1 Visa. My employer also has not paid me full salary as promised, which I am discussing with him.  Also, I got paid by check in January month for my December payment, so I am wondering does this apply for 2015 tax filing as I got paid in January 2016? I am wondering if there is a modifier or tool that will essentially undo a subsurf or subdivide command. A Monk threw three coloured darts, then spends one minute searching the battlefield, how many darts does the Monk recover?