Should have noticed this when these were first posted, but weaknesses in the demographic survey:
- No multiracial
- No Native America or Pacific Islander
- Hispanic is not a race, though I selected it (I am multiracial White and Native American)
- I have no clue whether people within two generations of my family have been diagnosed with specific mental illnesses and I'm not sure how widely that type of thing gets discussed outside of immediate family
- Was the above intended to only include direct ancestors? Because I took it to include siblings, cousins, aunts, and uncles
- You gave us the ability to say we're not interested in politics and not registered with a party, but did not give the ability to indicate we don't feel any particular allegiance to any of the listed named ideologies, either
On the weight loss survey if I lost a lot of weight due to a untreated medical condition, (then gained that weight back when the condition was treated), I'm guessing it'd be more appropriate to answer "no" to "have you ever lost more than 20lbs under a year" question, even if the answer is technically "yes"?
... I'm guessing "have untreated Crohn's Disease and c. diff for 6-8 months" isn't the sort of hot weight loss tips the survey is interested in.
I don't know what to answer in Survey 0, Demographics, for the psychological conditions where I'm both guessing I might have it (though not formally diagnosed) and I know I have a family member who's been diagnosed.
@Scott, I'll wait several hours before completing the survey in case you want to clarify which I should answer?
Survey 5 lists "Bupropion" and "Wellbutrin" separately. Is there some reason to do this, e.g. does Wellbutrin differ from generic bupropion in its effects?
Given that this is the summer, survey-takers in between (HS/university/grad school) and (university/grad school/fulltime employment) are especially likely to be in weird in-between states with respect to their occupations and residence statuses. Any preference on whether to answer as one's past or future self?
I have an identifier from a previous survey, I assume I should reuse that? The instructions don't explicitly say to use your old identifier if you have one!
Trying to do the weight loss one but I can't answer whether I have ever lost more than 20 lbs in a year as the side-effect of a medication or for some other reason because the answer is both. This seems to assume that if you have ever lost more than 20 lbs in a year, it only happened once. I suspect that is not the case for a lot of people and certainly isn't for me (for me, it's mostly because of sports, but at least once, it was actually due to medication side effects).
Survey 13 includes mescaline in the list of "psychedelic tryptamines", but mescaline is a phenethylamine, not a tryptamine. This makes it unclear whether people with experience with other psychedelic phenethylamines (2C-B, DOM, etc.) should include them in the "other" category.
Survey 19 has the polarity of the answers switch halfway through the survey with no notice (ie, 1 switches from “strongly disagree” to “strongly agree”). I suspect the answers after that point are going to be spoiled unless that’s fixed.
I'm a bit unclear on what I should put down as my work status since I'm a student but am also working at a convenience store. I guess I'll put student?
Completed all of the surveys that apply to me, other than the optical illusion survey which doesn't work. I consider this sufficient to boast/worry about, as per the post's suggestions
I find the political spectrum question tricky because I consider myself a centrist in my country but that would put me very much on the left in the US.
Survey 1's probability section asks for answers from 0-100 in percent, but requires a numerical answer with no other characters (and also no bounds on possible answers). This creates a lot of practical ambiguity if some people revert to answering on a 0-1 scale, since having all one's answers be less than 1% is not unreasonable.
I object to the omission of "Anarchist" as a political orientation. Left-anarchism is very much not the same as Libertarianism, and it's even more distant from any of the other options.
#21 does't load; the message is "A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?". I do allow cookies. I'm on MacOS using Safari, but at a guess this is the result of a bad URL rather than a browser issue.
For the mental health conditions in survey 0, I assume that if I've had a diagnosis for a condition before but wouldn't qualify for it anymore, that still counts as "I have a formal diagnosis for this condition"?
I have a question about the demographic question about depression. If I both believe I have un-diagnosed depression and have a family history of diagnosed depression, which answer do I give?
The question is:
"Depression
Regarding clinical depression...
(a) I have a formal diagnosis of this condition
(b) I think I might have this condition, although I have never been formally diagnosed
(c) I have family members (within two generations) with this condition
(d) I don't have this condition and neither does anyone in my family"
b and c both apply. Some other demographic questions have this issue as well.
The questions on the Metaphysics survey are so hard to respond to that I gave up. For example, the question: "How certain are you that racism is incorrect?" Racism is an act, not a proposition! How am I supposed to answer that in a meaningful way? Many of the other questions were ambiguous or didn't have an option for the answer I would actually give.
I wish #4 had at least one free form text input field, and I suspect my answers will appear inconsistent to its author.
To summarize what I would have said: I know a lot about how most digital technology works internally, but experience the user interfaces as having been designed to be hard to use, hard to learn, and most likely to intentionally exclude older users, as well as those with a number of disabilities.
I no longer consider myself "digitally literate", since I can barely perform basic functions with the latest cell phone style of interface (no text, lots of icons, UI elements invisible until you make the right gesture at some unmarked spot on the screen, no help, no user's guide)
I did all the ones that applied to me except for two, where the formatting was counterintuitive for me and I kept making mistakes, so I didn’t want to submit with errors
That was extremely fun, but I have some questions.
On the Metaphysics survey, it went from page 1 of 3 to page 3 of 3. I checked, going back and forth, and it came up the same. Is there (a) a missing page 2 (b) ha ha, deliberate error, this is a metaphysical question?
On the Weight/Gain loss survey, it gave a great selection of "is this why you lost/gained weight" but nothing as simple as "went on a diet" or even "Other - lost weight because of this thing, gained weight because of that thing" which I do think is a flaw. I lost (and gained back) weight over a two-year period, but it wasn't because of "broke up with a boyfriend" etc. reasons
On the Covid-19 Impact, I skipped the wages question because (a) I can't remember exactly what I earned back in 2019 (b) I certainly can't be bothered trying to work out "if I worked X hours per week at Y rate per hour, what is that over a year?" and (c) none of your business, toots: if you want to know did I lose money during the pandemic, just ask that!
Question: Does everyone have a race? It's a social construct after all. I'm sure the american society could possibly assign one for me, but I've never been there. Does it only go by apperance? Then why would we need another word for it? It also seems different from the concept of 'people whose ancestors came from the same place look somewhat similar'.
I know about the falsified scientific theories about different human races, but you probably don't want such bogus categories in this neat little dataset.
I don't really want to describe where my family and I come from and live, because then people might try to invent a race for me instead of answering the question ;)
Survey 23, "Metaphysics" leaks the survey results after submission, meaning you can just go and look at everyone's answers, including their identifier. https://i.imgur.com/Gdo8Sa3.png
My quibble on the demographics survey: it treats "having a mental illness" and "not having a mental illness" as a binary. What if I was diagnosed with the condition in the past, but no longer have it?
Completed all that apply to me, except 21 which is broken as others have reported already. Officially bragging.
"Metaphysics" was very fun to do, although the two math problems seem exactly equally hard to me.
"Understand your own beliefs" was also nice because I was surprised to actually feel represented by some answers. Spotted some conflict-vs-mistake-theory in there :)
The one about weight gain / loss left me somewhat confused. Not much happened directly before my birthday in 2018, and I had no idea why it was relevant.
"Health-Related Quality Of Life" was not explained very well (too wordy and therefore more confusing that it needs to be), but the research topic is super interesting and I hope something relevant comes out of it. I had to concentrate a lot while answering, though.
Survey 9 doesn’t work. After submitting a few pages of responses, one page doesn’t let me submit, instead telling me “ The question has changed. Please review your response.” This happened several times.
Chiming in yet again to say that “country that you most identify with” means nothing, and that you should split it into “country that you currently live in” and “country where your primary cultural background is from”, or something similar. I will continue to complain about this on every survey that has this question.
Also seconding everyone who complains about the race question. I already hate the US system of racial divisions, but if you’re going to use it, the bare minimum should be to allow for multiple selections and to have a place where you can write-in an “other” option.
ok enough griping time to fill in two dozen surveys
>Automation and technology is going to rapidly take away many people's jobs.
I always hate just answering with either agree/disagree on questions like this. I feel like most people's interpretation if I agree will be that it's bad and might generate a headline like "85% of economist are worried that automation is going to take away people's jobs."
How I want to answer that question is "Yes they will and that's a good thing. Here's why...."
Re customer satisfaction survey --- I read ACX via the RSS feed. So if the email letter stopped I would keep reading it but I wouldn't go to the website.
On the demographic survey, it should probably ask how long people have been reading ACX, not SSC - or maybe mention both. But some new ACX readers might not even know what SSC is.
I guess maybe I should have skipped the psychadelic survey entirely, but it was weird that there's no way to answer "I tried one and nothing happened".
I frequently find myself confused by the questions in the "Metaphysics" survey — sometimes none of the possible answers seem correct at all, or else the question is based on a false premise. Oh well; guess I'll just try to pick the least incorrect option, and skip the worst offenders. Hopefully the results aren't announced without describing what the options were.
I can't tell if "how long have you been reading ACX?" Is supposed to be a trick question. The first post was (just) less than a year ago, but I've been reading Scott's writing for at least a decade, so I assume I'm supposed to answer more than 2 years?
> When you dream, does it occasionally happen that the *you* is a different person from your waking self? If so, is that person male or female?
I'd say in the last 5 years I've had a dream that I remembered upon waking 10 times, and even at that never to the level of specificity that would allow me to answer these questions.
I answered everything I could and it was fun! A few comments:
- Americans (as in the USA, but "people from the USA" is a mouthful) obsession with race is downright bizarre coming from a European point of view. Maybe a survey about that would be interesting?
- I've seen the word "community" used and thankfully it was explained (a city, town). I often hear it, and feel like it's also a very American point of view. For me, a community would be people you are close to, share something with. Family and friends mostly, maybe a few colleagues? But the question seems to imply that it could be two million people. Does that word has a strict meaning?
- Behavior during COVID pandemic: some questions were necessary to answer while a question before essentially made them moot, so I had to reply with bad data.
- Weight gain/loss: no mention of the COVID pandemic? That seems like a big oversight.
- 1 to 5/1 to 7/strongly disagree to strongly agree questions in general: I'd like a metasurvey about this. I have a hard time determining if I feel more like a 5 or a 6 on most issues. Does "strongly disagree" means "equivalent to the thing with you disagree the most", or "equivalent with the thing that you saw people disagree the most"? For example, I think most people are more vindicative about issues than me. Does that mean that I should only answer in a reduced scale, or should I use the full scale. Are there studies in this area to determine if this kind of question is meaningful even if it's only relative? Is it a case of the perfect being an enemy of the good?
- again a metasurvey proposition: I might be the exception in this community, but I don't feel strongly about most moral issues. Do we have any data on that?
- Understanding your beliefs: Is it bad to be a homosexual: I don't know if it's on purpose, but it's confusing to answer. I don't have anything against homosexuals in general, but I've endured some suffering because I was one that non-homosexuals haven't endured.
On the Covid-19 Impact survey, for the question "During the COVID-19 pandemic, did your employment require in-person interaction with the general public?", it doesn't give the option to indicate that it's not applicable because I was not employed. I selected "unsure / prefer not to say", but I see that the survey creator is active in this comment section, so perhaps they can give me some guidance.
My answers to some of the sex questions are completely dependent upon how in on it the other person is. Like, touching a stranger at a sex club where everyone there is like "yeah, this is where we will get touched by strangers" is completely different from touching a stranger on the bus.
I literally don't fall into any of the political buckets. Normally I'd say this is a matter of "I'm so special" syndrome but when you can't agree with at least half of any of the combined characteristics, what are you to do?
Small point, but in Survey 0, I interpreted the question "How long have you been reading ACX?" to include SSC as well. I assume this is what was meant, but some people might take it literally.
I'm sure you're inundated right now with people asking questions about the survey, but are the Life/Job/Social/et-al. satisfaction sections gone? Those results always seemed the most interesting to me.
Like, lawyers reporting higher average life satisfaction than software engineers was an unexpected and interesting result that kind of cut against the voguish archetypes of the unhappy, stressed lawyer and the chill, relaxed software engineer.
If they are gone, maybe they can make a comeback in future iterations?
Filled out most of these surveys, but the only one that caught my interest was survey 23, metaphysics. I would love to see the data on correlations between answers to those questions, but there is no information of who the survey is made by, or what the data will be used for.
22 More Psychedelics the questions about health insurance are less meaningful if you live in a country where that's not the main way people get health care. (As is the case for much of Europe). I answered as about whether whoever the healthcare provider was should provide it which I assume was the intent
In "behaviour during covid" the question "If any of the following influenced your perception that the pandemic was easing up or ending, please check all that apply: *" seems to list a bunch of things about my personal experience and feelings. Which might effect how much I follow the lockdown rules, but are a separate thing from whether I feel it is easing up or ending
In the "can society be wrong about words" question on the metaphysics survey, the question sheet only showed me two options but the survey results page shows four. Also, that was the only survey (that I took) which had a "see previous results" link at all after submitting – was that intentional?
Regarding #14, businesses being "larger than some countries" isn't saying much in a world containing Vatican City and Andorra and San Marino. Yes, I'm totally fine with a company being larger than Andorra.
The rhyming survey is labeled as "targeted at non-native English speakers", but the survey doesn't actually seem written with that assumption. "English" is an option for native language, and "US English" is given as an example on the dialect question. It looks more like "we're studying how non-native speakers differ from native ones" which means that responses from native speakers are needed in order to have a control group. The label may be discouraging some native speakers from usefully participating.
At the end of the cryonics survey it says "Please check reddit.com/r/cryonics in January 2020 for survey response data. " So unless the cryonicists are branching out into time travel I think there's a typo
Some surveys don't ask for the ACX ID; mentioning in case this is not intentional. (Sorry, did not keep track which, was focussed on going through all of the surveys.) I added it in a freeform comment field for one of them, but that wasn't always possible.
I just reused the SSC ID I gave myself in a previous reader survey a few years ago, by the way. I'd saved it in my password manager and there was no reason not to reuse it, really.
Thanks for the surveys, everyone who made them! :)
Scott: It wasn't clear to me how to record a historical rather than a current diagnosis in the mental health questions in the demographic survey. I can see in the comments other people had the same issue. Maybe change the wording to "I currently have or have had a diagnosis" next year?
Other survey writers: please understand that not everybody is from the US, and phrasing your questions assuming everybody is American makes them hard for us to answer, affects your data quality, and also drives many of us slightly insane. For example, if you want to know my opinion on right wing people, that isn't the same as my opinion on American Republicans.
Did 'em all other than the non-functioning optical illusion one and those (if any) that were demonstrably intended only for demographics excluding my own. The metaphysics one was repeatedly infuriating, demanding that I choose between two options neither of which I could honestly endorse.
Survey #3: What do the colors mean? There's red, yellow, green, and blue, which are supposed to represent some of different health status, but none of those are mentioned in the survey text.
Question about how to answer the "work status" question on the reader survey: There is an "Academics (teaching side)" But there is no "Academics (research) side"
I work in a university lab doing research. I do literally zero teaching.
My best guess is either non-profit, or "governmental work" since all of our funding is state or federal grants.
The "neither does anyone in my family" for the mental health questions is ambiguous. Do you mean genetic family (ie you're interested in the genetic connection & no one else counts) or family in a broader sense (in-laws can give you social experience with people who have the condition but won't pass it on genetically).
As a point of self-comparison, I seem to be getting half a point more politically right-wing each survey. I wonder if this'll keep going, and (if it does) if I'll move far enough to actually vote for a right-wing party by the time I'm actually able to vote somewhere.
I was completely unable to load the Optical Illusion survey. Error message "couldn't find cookie result for study result ID 31188. Are cookies allowed in your browser?" Yes, yes they are. Am I trapped in a metaphysical anomaly?
"To what extent do you find yourself behaving differently when in a group of all/mostly women compared to when in a group of all/mostly men (which are otherwise equivalent to you)?"
I don't know, I haven't been in such a group in aeons, unless you count the internet, which makes gender harder to track.
"You wake up one morning with your body changed dramatically in one way, but otherwise as healthy as before. Which bodily change would feel the strangest/most disruptive to your perception of self?
30 cm taller than you were before
With the opposite sex organs (male to female and vice versa)
With a drastically different skin color (e.g. black to white and vice versa)"
Is this about swapping primary sex characteristics only, such that, say, a cisgender person would wake up with a body comparable to a transgender person of their gender who was unusually lucky with HRT results but had no surgical interventions, or is this such that a cisgender person would wake up with a body comparable to a cisgender person of the opposite sex? I am tentatively assuming the latter, but the phrasing is ambiguous.
Finished all the surveys that pertain to me, other than 3, as it was asking me to imagine things I cannot imagine, and to evaluate them in a framework which is utterly nonsensical to me. Answering surveys can be quite fun, it seems!
I think the "intellect significantly enhanced" option in 3, while a decent specification for an ordinary sample, was not a great choice for ACX; it specified as IQ 140-150 and IIRC the average IQ here is nearly that, so it's going to return a whole pile of "no use" and "negative use" results.
Half way through "Attentional Focus and Performance Orientation" it puts "I agree a lot" on the left, with the lower numbers, and "I DISagree a lot" on the right with the higher numbers. This is opposite to basically all the other surveys and most surveys I've taken. It was jarring when I noticed, had already answered one question without noticing. I expect many people will answer those questions incorrectly. I suggest switching it around ASAP.
"Scandinavian countries: heavily-regulated market economy" - this seems like a gross over-simplification, one that shows up in annoying political debates and makes them more annoying.
Optical illusion survey worked for me and my eyes hurt and I want to slap whoever came up with it. It went on *forever* and it didn't even tell me how many more I'd have to do before I could stop without it chucking my data!
The incest survey is hard to do because it's hard to let go of doubts regarding consent and power dynamics and assume everyone is a fully independent and consenting adult as the survey asks you to do.
Okay, I got through most of them, skipped a few that weren't aimed at me or were very difficult to understand. I will accept your praise slash concern.
The personality survey bundles together things that I feel shouldn't be bundled together at all. For example, I do feel "down to earth" but not unimaginative. I'm ethical, but not unforgiving. I answered natural on these question though cause if they were separated i would give opposite answers.
Does the Optical Illusion one record my results if I stop partway? I eventually gave up and quit. I'm not even convinced that it was ever going to stop.
I think it can discourage many people that the survey is hosted by Google Forms. Even though you do not type in a name, Google has numerous way of tying the answers to your identity and will have them stored for god-knows-how-long. Given the personal nature of some questions, it seems to have potential for abuse. Given how trivial it is to self-host a simple survey form, it would be the respectful thing to do w.r.t. your users.
21 is a very unpleasant experience and takes too long - the visual illusions made my head dizzy and my vision blurry. I tried keeping at it for as long as I could, but I just couldn't finish it.
Do you strictly want this to be a reader-only survey? What if this blogpost is someone's first ACX read - would you consider them as being an 'ACX Reader'? I want to let readers of my newsletter know about this survey but I do not want to adulterate your survey with non-readers. Let me know.
I don't know what to enter in the "work status" question. I work for a UK University. Like almost all UK universities, it is an independent but largely publically funded institution. It is technically a charity. So it's not for-profit, nor technically government, but it's not normally perceived as non-profit.
In the UK, we'd normally describe universities as "public sector", so I'm guessing that translates to "government" in USA-speak, even though we're not actually part of the government.
Regarding the ACX survey (24), going through it clarified my thoughts about the newer blog and Scott. I would definitely pay for a published compilation of the old essays & fiction, but I don’t read this blog with the intensity I read SSC.
Nowadays it’s a casual read. I like to read professional writing that’s not in my field, and I read In the Pipeline for a similar reason. SSC essays had a style wit and presentation that I really liked.
General demographics "profession" subtype computers: need to choose either "practical: IT, programming" or "AI". I object to the claim that AI is either not "practical" or does not require "programming" - I do practical programming of ML systems, which is what almost everyone means by AI. Does the survey mean "AGI hard research"?
In survey #4, "Digital Literacy", it asks you whether you're familiar with a list of technical terms. One of these is "proxypod", which is presumably a fake term thrown in to check if I'm just clicking "yes" to everything - but it sounds to me like it *should* be a brand name for a low-power VPN server.
In survey #0, "General Demographic Information", it asks how many children you have. For men, in particular, it's possible to be uncertain of this, but the field requires a specific number rather than a range.
Hey, this is kind of a weird comment, and I don't want this to sound preachy or anything but I am a little disappointed in the categories when it comes to the profession tag.
I currently work at an Amazon warehouse as an associate. Not one of the full-blown fulfillment centers, but one of the smaller delivery stations. While I have a pretty firm way of how I would describe my job to others: "Warehouse Worker", there is no option like that here. Okay so I should check "other blue collar." Fair enough.
But what if I still had my old job as a Dollar Tree cashier? What would I have put then? It is neither white collar nor blue collar under traditional definitions, yet not only do a majority of the people I know work in the retail industry, according to the Bureau of Labor Statistics, it is still one largest industries in the country in terms of people it employs. Keep in mind that Walmart, Amazon, and Kroger are the largest private employers in America last time I checked, and it's no coincidence that all of them have large retail/grocery components.
And I understand that a majority of readers who read this blog are more academically/IT inclined and you would want to capture the diversity on that front as much as possible, but I think you lose something when you have three different categories for IT and only two broader "Other" categories for what a majority of working Americans actually do. There could be interesting differences between the retail workers and warehouse workers that now won't show up because of the broadness / ambiguity of the definition. I know I have a lot more back pain working at Amazon now than I ever did at Dollar Tree. You can't even write in an answer to clarify the "other" so it is a difference you may never know.
Again, sorry for the rambling, I've been a fan of this blog and its predecessor, but that element of survey has always rubbed me the wrong way. I understand that it may be too late to substantially alter it now, or there might be space limitations, or other good reasons to keep it the way it is that I haven't thought about. Whatever the case is, I just thought I would mention it.
All surveys complete! Except the optical illusion one which wanted a cookie; I see others have had the same problem.
How to make your ethnicity surveys Brit-friendly, by a Brit:
* "Asian". In North America that means East Asian unless you write in South Asian. In Britain it means South Asian unless you write in East Asian. So a South Asian Brit who clicks Asian will find themself categorised with the American East Asians instead of the write-in South Asians. I know you don't want 10 options plus a write-in box, but I'd consider splitting those out.
* "Hispanic". In Britain, Hispanic is a fancy word for a (typically white) person from España - c.f. Gallic, Teutonic. "Non-Iberian Hispanic" is a useful modifier, or just call it Latino (which also avoids the arguments about whether Brazilians are Hispanic).
* "White/European". Try to avoid this one. Millions of people of colour live in Europe and are proud Europeans, and with Brexit and the racial abuse of England's non-white footballers at the Euro Championships being hot topics, it's a bad time to equate not-White with not-European. "White/Caucasian" is better; "White" is perfectly good.
Unrelated to race: the average Brit doesn't know what an FDA is. I would hyperlink FDA and put "or your country's equivalent".
The average Brit also doesn't have interesting opinions on American politicians or the state of American discourse. Can you let us know whether to answer for Britain or to skip those questions?
One of the surveys had Agree on the left and Disagree on the right such that "1" was strongly agree and "9" was strongly disagree. I wonder how many people are going to have read the words above the numbers?
Also the metaphysics survey was way to hard for 1:30am
The "Family Demographics And Homosexuality" survey is confusing because it only asks for ASAB, not gender, and then it asks "Are you exclusively homosexual?". I'm not sure what is meant by "homosexual" in such a context.
So I did a bunch of these yesterday. Early this morning I have this long evolving dream. I’ve taken a new job and my new employer is administering these these endless seemingly pointless multiple choice tests to see where I’ll best fit in with their teams.
Oy. This is never going to replace my favorite. I’m the captain of a pirate ship, straddling a canon. My crew is the University of Minnesota women’s volleyball team. I can’t help but think that one means something…
From 'Much less than normal' to 'Much more than normal' -- does that ask the personal 'normal' (healthy, i.e., 'Me before X'), personal average, or a population average?
I had one of those awkward both/neither reactions to the incest survey.
I want a blanket prohibition on sibling incest. I don't want the legal code to have to get into the minutiae of which sex acts are illegal under which conditions; that's just an invitation to lawyers to find loopholes. I also think sibling incest is a *bad* choice, and don't want people advocating it any more than I want them advocating smoking.
But under the exact conditions specified in the survey, I would issue guidance to the DA/CPS that it isn't in the public interest to prosecute anyone. So the "restrictions" wouldn't actually restrict anything.
It would have been nice to be able to give direct feedback about a survey to the survey author. I realize I can comment on it here, but don't know if that will get through. For example, after answering several questions on the "Health Related Quality of Life" I felt the comparisons weren't specific enough for me to give accurate answers (so I decided to bail out rather than skewing things)
In 21 (optical illusion), I see the red cross and then it all goes black, which seems to me like unintender behaviour. I'm using Brave browser. I have already answered the first two pages of questions, so it's annoying to test with another browser.
I said I was "hispanic" given the options, but that didn't feel like a "race" question to me, but nationality. The alternative being "white (not hispianic) [where hispianic is how americans call people from certain places]".
I found the 'Personality' survey very frustrating to take. Most of the questions seemed ambiguous at best. Do they want us to answer based on BOTH words, or based on EITHER word? For instance, 'active' and 'talkative'. Do they want a response to 'active AND talkative' or a response to 'active OR talkative'? Most all of the questions suffer from this defect. What if I'm talkative but not active? What if I'm carefree but not amoral? The author apparently feels that words like 'emotional' and 'anxious' are synonyms or something. Not true.
I quit 17 minutes into the Optical Illusion test that supposedly takes 10-15 minutes. I hope it at least saves the data part way.
That test *seriously* needs to be a) much shorter and b) have a progress bar. After a while, I started to suspect that it was secretly a test to see how long keep would keep doing the same thing over and over before giving up.
I am probably skipping the survey this time because of the trivial inconveniences that exceed my threshold. Instead of filling one long form, it's "create a weird id, copy paste it, keep it some place for every other form, insert it into the next form..."
I wish it was more like "this cookie will be used as your default id for the survey, you can edit/replace/delete it as you see fit, should you care enough", then each form would end with a link to the next one, id-preserving.
The ACX Customer Satisfaction Survey made me think about why I considered becoming a paid subscriber to ACX, but chose not to (so far). I certainly enjoy ACX, am a paid subscriber of other things, and could easily choose to subscribe to ACX.
After thinking about it, I'd say the main reason is actually Scott's repeated downplaying of the importance of subscribing, and of its benefits. It's basically always presented as *super* optional, with an underlying message of "don't feel bad if you can't subscribe, it's really not that important, all of the good articles are free anyway".
I understand why it's presented that way, to avoid some people's scrupulosity issues, but it's honestly so much of a downsell that it makes paid-subscribing sound like an unattractive option. It sounds like "if you *really* want to throw away your money, then be my guest, I guess".
I am sure this was discussed to death already, but I lost the memo. What does Hispanic mean and how is that in the race category? I am Argentinian and so I don't identify as white (mainly because all white countries have decided we are not white, so I follow suite here and do not want to identify as white) but neither do I identify as "hispanic". In the actual hispanic language, that means Spaniard, which I am not. In the unitedstatezen language, as far as I know, hispanic means "Mexican looking", which does not apply to me either, neither by personal identification nor, I believe, by the standards of the natives that came up with that word (I've had 4 Unitedstatetenzens, as well as 7 Europeans and dozens of Chinese, tell me straight to my face: "ohh, so you are Argentinian but also obviously (never understood why is that obvious) not Latino/ not Hispanic, what race are you?"
I chose White, at the end. Does that mean that all my survey answers will be read as belonging to some anglosaxon protestan puritan or similar? Probably not very important, but in my mind that would disqualify the results at least a little bit.
(I really would like to know what Latino/Hispanic is suppossed to mean and why I so obviously do not belong in that to the people that came up with those words. Actual Latinos/Hispanics do not use those words nor do we feel we are a "race")
Just here to boast - did all of the ones I was demographically qualified for... except the optical illusions one, I couldn't get that one to work (chromium, Linux).
I've been reading OB/LW from way before LW even became a thing. I feel like a large part of the impact that cryonics has made on rationalists stems from its discussion there. My life doesn't revolve around cryonics, but I'm sure I've read most, if not all, of the posts about cryonics on OB and LW. **I can't recall ever hearing the term "biostasis"!** Using Google's site search, there doesn't seem to be any mentions of the term on LessWrong until this year...
Someone tell Scott that there needs to be a "got pregnant and then gave birth" option on the weight gain/loss survey. Pregnancy and childbirth are the only reason I've ever gained or lost weight, but my weight history looks completely bizarre without noting "gave birth then."
I don't understand why pregnancy wasn't mentioned/asked about in the survey since it is probably the single biggest reason why people suddenly gain and then lose more than 20 lbs in a short amount of time.
I completed all of the non-targeted surveys except 7,(which may merit a "targeted at people who have ever meditated note, I quickly grew bored with skimming through long lists of things that read as nonsense to me just to check the "no" button), and 21. (Took way too long and felt like no actual end was in sight, beginning to agree with the suspicions posed by other commentators that the whole thing was an exercise in testing patience.)
Of the set, I thought 16 was the most interesting. I'm looking forward to seeing investigations of those results, and what the intent of the survey was.
Survey 5 "targeted at people with depression/anxiety" -- The questions all relate to symptoms and treatments for depression. It instructs a person with anxiety to word-substitute anxiety where it says depression throughout the survey.
I don't think doing that is going to yield meaningful data about people with anxiety. Anxiety and depression not uncommonly co-occur, but they are different things with different symptoms, different coping behaviors, and different treatments.
i'm not sure if it was there and i just missed it, but i don't think the meditation survey asked about *when* you experienced the weird religious-type experiences it's asking about, or the context in which they happened? i'd venture to guess people with, for example, charismatic christian backgrounds or schizotypy would have plenty of experience with weird religious-type experiences.
The weight loss survey included questions about life events that looked like they were trying to approximate something like the Holmes and Rahe stress scale, but it was missing the most impactful questions (e.g. "death of a spouse", "divorce", or "imprisonment"). An oversight, or did the survey had other things in mind?
So i gave up on the optical illusion one because it seems to go on forever with very similar or identical optical illusions. Did anyone actually complete it?
I ran into on the first trial of optical illusions after the practice trials. In case this helps debug: "Error in Fixation Cross (eventAPI.js:101): during event after:end : EncodingError: the source image cannot be decoded."
I am having trouble with the COVID questions. These seems to be no option for "I absolutely did not care at all". I have never been even the slightest bit concerned that I would "get" COVID or be harmed in any way by it if I did. Biologically, it doesn't even exist as far as I am concerned. So questions like
"During the Covid pandemic, which worried you more (choose 1):"
Are meaningless.
Did you model these after the tautological and undecidable "COVID questionarres" that have to be filled out all the time for insurance coverage?
-- "In the last 14 days, have you possibly been in contact with a person who may have been exposed to COVID?"
Uhhh...how can I possibly answer that except with "Of course, that is possible"?
I feel I may have to punt this questionarre because I can't answer accurately.
__________________________________
Let me go further, because I feel like my perspective is unique and might have some value to the community, which is why I hope it can be capured in the survey(s)
This survey is so much worse than previous surveys that I failed to do more than the first two and gave up halfway through 3. I put this down to the hassle of the user ID, having each part separate without indicate who should fill them in (2 is only aimed at men),the awfulness of the questions in 3 (as a disabled person they made me feel physically sick) and the overall lack of sensitivity (how hard would it be to include a non-binary option for instance). I've lost my user ID now so I won't be doing more but I though you might like to know, as the survey was never this bad before
Regarding #19 (Attentional Focus And Performance Orientation):
The second half of the survey gives a 1-5 scale with 1 labeled as "agree a lot" and 5 labeled as "disagree a lot". This is the reverse of what I would normally expect, and I almost didn't notice that higher numbers didn't indicate stronger agreement. I answered according to the provided labels, but other people might not notice the change or might assume it was a mistake and answer the reverse.
The question about health reminds me of https://xkcd.com/883/ -- it is easy to remember all things that are wrong or suboptimal with my body, but when I start to think about all the problems I *don't* have, or just list all organs in human body I know about and then consider how many of them never gave me a reason to complain, suddenly I feel quite healthy.
Now sure why should my weight be correlated to what I did in 2018, but whatever.
Note to survey authors: if you make a scale with both numeric values and verbal descriptions, please make "less" correspond to smaller numbers and "more" to larger numbers, otherwise it is a bit confusing. ("How much do you like kittens? 1 = a lot, 7 = just a little")
If your survey is long, please split it into pages of similar length and always say "page X of Y". It sucks when I am at page 10 and feel quite bored, and I have no idea whether there is 1 more page waiting for me (in which case I would rather complete the survey) or 20 more pages (in which case I would rather quit).
I'd be curious to find out what that "optical illusion" survey was *actually* testing for. Clearly, the "illusions" are supposed to distract us from something... but from what ?
Whoever wrote Metaphysics, your question about intersex conditions is *awful*. XXY/Klinefelter's is only "ambiguous between male and female" if you know absolutely nothing about sex chromosomes or their aneuploidies. I recognize this misconception is fairly common, but that doesn't make it not "so immediately dumb that I considered closing the tab instantly, because if all these questions were written with that understanding of their concepts then no meaningful results could possibly come from them".
FYI, Qualtrics surveys are not accessible over Tor, so on the more sensitive topics you may lose some people who would only answer from behind an anonymizing proxy.
On the other hand, I suspect a goodly percentage of the survey-takers could actually be uniquely identified by the information requested in the #0 demographics survey (though perhaps not actually located by it.)
For next year, it may be good to set a norm for Likert scale directions (i.e. either always have agree on left/disagree on right or vice versa). There's some variance across surveys and have realized a couple times that they were not set up the way I thought.
Wow, I had a really hard time understanding what was being asked of me in the hypothetical health states questions. I'm not sure how I would write clearer instructions for it, because I'm not entirely sure I get it still. The instructions almost certainly need to be shorter.
For me, the optical illusion test took a little over 30 minutes, plus breaks for eyestrain. Regardless of what some commenters suggest, there IS an end to it, it's not testing just how willing you are to keep doing a repetitive task. However, it does save the data even if you don't have the time to complete it.
I wish that the optical illusions survey (#21) had a button for "faded in completely, but I can still see/feel the rectangle border, so not an uniform image". Because in that case, you always have to wait while staring at a dot, and it requires a LOT of willpower. If I could just press A/B or something, that would make it a lot easier.
Also, I accidentally noticed that the rectangle border becomes more apparent when you slightly move your head while still keeping your eyes fixed on the red cross. I tried not to do it and be "good".
One, quite wordy and convoluted. Two, actually bloody hard! Had to think quite a lot about what appropriate answers might look like. Three, my priorities seemed to differ wildly from the author's.
Really looking forward to reading the results, the goal was one of those Basilisk extremes, monster cyborgs that no normal human or society would ever sign up for (hopefully), but there's more insight in it beyond the original idea.
I wonder how many other outliers are like me, and simply high school graduates, and what they're like. Whether they're all grumpy and cynical like life beat into me, or if I'm just an outlier among outliers. Or maybe they all have other mental illnesses like ADHD or other heritable disorders. It's really difficult to find outlier-based surveys where I might glean some insights into "my demographic", as generally "high school and some college" results are lumped together into "poor and ignorant" instead of "chronically curious but unable to pursue academia". Will you be doing a separate analysis of such persons? Are there even enough of them? Please do, I would like to know just to see if my psychology correlates to this or other factors.
I started two surveys and had to quit both. My (undiagnosed) ASD doesn't allow answering questions where I know my answer will be interpreted incorrectly.
For whomever is responsible for survey #3, you wrote at the bottom of the survey:
> It would be very helpful if someone could suggest a way I could randomly generate hypothetical health states in an online survey (i.e. randomly complete the five possible states in their three possible levels). I understand enough 101-level coding to write a program to do this on my local machine, but I don't know how / where I could host it in a way that would be suitable for this sort of survey.
Basically, I'd generally suggest creating a web application and hosting it on (deploying it to) a VPS – a virtual private server. There are some other possibilities for what's effectively free hosting too, and you _could_ provide public Internet access to a program running on your own computer (and for free), but a VPS is the most general way to host a program publicly. (But keep in mind that this can be arbitrarily hard – the Internet is very much a 'densely populated ecosystem' and a publicly accessible server will be 'under attack' almost immediately after accepting incoming public Internet traffic. I wouldn't overly worry if it's only going to be live for a relatively short period tho, e.g. a few weeks, or even a month or two.)
I'm open to answering any other related questions here too.
The metaphysics survey having "option 2" and "option 1" in that order is really aggravating. It might be random don't make me guess how to enter a right answer.
In the optical illusion survey, I often found that the images looked uniform *before* I was allowed to press the space bar. I fear that this design choice (to only allow perceived uniformity after a certain time) may significantly impact your results.
To beat the dead horse I mentioned in a previous comment on this post, I found myself, SO many times, wishing for another option/answer to a survey question, or even just an 'other' option/answer with a text box, for at least _some_ way to escape the provided 'framing' and challenge the assumptions seemingly baked into to the questions! Having now just been subjected to that, over and over too, I hope I can remember that feeling every time I ever read or hear about any survey results ever again.
Rough showing for Christian Democrat political views. It was one of the dominant political parties and ideologies for multiple European countries throughout the 20th century. It is still in power in many countries today - Merkel is a Christian Democrat, for goodness sake.
But for this survey, that viewpoint does not exist. You can be a U.S. Democrat, or a Scandinavian socialist, but a Christian Democrat - nope, not on the table.
This is perhaps reflective of the readership and survey makers, but is not progress for the world in my view.
I was 80% through the autogynephilia survey when I clicked on "Previous" to check something in a previous page and was suddenly back at the first page with all fields empty. Was my data never saved at all and I'd have to start from the beginning, or was my partial answer saved and I can just enter my id again, maybe put this in the first other field, and continue where I left?
An interesting thing to measure would be, for each user ID, how many questions they answered: there might be some correlations between that and other things.
On my phone, for questions like the rhyming survey, the higher values are hidden - I see only options 0, 1, 2 in the rhyming survey, I need to be aware that I can scroll to higher values, or all answers will end up 2 or less. Maybe, something to control for?
The Survey #19 is a bit weird with how the responses are presented. For the first part of the survey "often" is on the right and "never" is on the left. So the more that statement applies to you the more you select your answer on the right.
After about half the survey it changes and "I agree with this" is on the left and "I disagree" is on the right.
I had to correct a bunch of answers because I caught it in time but I could imagine that some people answer the second part of the survey with the opposite to what they actually mean.
I took the 19 non-targeted surveys, 1 targeted survey that I was the target of, and accidentally 1 targeted survey I was not the target of by going in order and not noticing it was targeted/not reading the parenthetical after the link. Oops. There's at least 1 incorrectly targeted response to the Rhyme survey.
I'm half Japanese, quarter French, and quarter German. How should I answer the race question?
Should have noticed this when these were first posted, but weaknesses in the demographic survey:
- No multiracial
- No Native America or Pacific Islander
- Hispanic is not a race, though I selected it (I am multiracial White and Native American)
- I have no clue whether people within two generations of my family have been diagnosed with specific mental illnesses and I'm not sure how widely that type of thing gets discussed outside of immediate family
- Was the above intended to only include direct ancestors? Because I took it to include siblings, cousins, aunts, and uncles
- You gave us the ability to say we're not interested in politics and not registered with a party, but did not give the ability to indicate we don't feel any particular allegiance to any of the listed named ideologies, either
What's with all the gender Ideology stuff in the 0: General Demographic Information survey?
At that I'm out ⛔
It looks more like an idealogical purity test than a demographic survey.
Sex Not Gender ♂️♀️
On the weight loss survey if I lost a lot of weight due to a untreated medical condition, (then gained that weight back when the condition was treated), I'm guessing it'd be more appropriate to answer "no" to "have you ever lost more than 20lbs under a year" question, even if the answer is technically "yes"?
... I'm guessing "have untreated Crohn's Disease and c. diff for 6-8 months" isn't the sort of hot weight loss tips the survey is interested in.
Survey 21, optical illusions, would not open with the error "A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"
21 seems to be broken - "A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"
I don't know what to answer in Survey 0, Demographics, for the psychological conditions where I'm both guessing I might have it (though not formally diagnosed) and I know I have a family member who's been diagnosed.
@Scott, I'll wait several hours before completing the survey in case you want to clarify which I should answer?
Survey 5 lists "Bupropion" and "Wellbutrin" separately. Is there some reason to do this, e.g. does Wellbutrin differ from generic bupropion in its effects?
For family members with mental illness, what is considered family? Sister? Uncle? Cousin? Or strictly two generations of ancestors?
Given that this is the summer, survey-takers in between (HS/university/grad school) and (university/grad school/fulltime employment) are especially likely to be in weird in-between states with respect to their occupations and residence statuses. Any preference on whether to answer as one's past or future self?
I have an identifier from a previous survey, I assume I should reuse that? The instructions don't explicitly say to use your old identifier if you have one!
Alert: Survey 2 doesn't ask for user ID
Trying to do the weight loss one but I can't answer whether I have ever lost more than 20 lbs in a year as the side-effect of a medication or for some other reason because the answer is both. This seems to assume that if you have ever lost more than 20 lbs in a year, it only happened once. I suspect that is not the case for a lot of people and certainly isn't for me (for me, it's mostly because of sports, but at least once, it was actually due to medication side effects).
Survey 13 includes mescaline in the list of "psychedelic tryptamines", but mescaline is a phenethylamine, not a tryptamine. This makes it unclear whether people with experience with other psychedelic phenethylamines (2C-B, DOM, etc.) should include them in the "other" category.
Survey 19 has the polarity of the answers switch halfway through the survey with no notice (ie, 1 switches from “strongly disagree” to “strongly agree”). I suspect the answers after that point are going to be spoiled unless that’s fixed.
What's the deal with user ID? Where can I get one? Is it necessary?
For me, survey 21 says: A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?
I have cookies enabled.
I'm a bit unclear on what I should put down as my work status since I'm a student but am also working at a convenience store. I guess I'll put student?
The digital literacy survey has some typos and grammatical errors.
Completed all of the surveys that apply to me, other than the optical illusion survey which doesn't work. I consider this sufficient to boast/worry about, as per the post's suggestions
I find the political spectrum question tricky because I consider myself a centrist in my country but that would put me very much on the left in the US.
Survey 1's probability section asks for answers from 0-100 in percent, but requires a numerical answer with no other characters (and also no bounds on possible answers). This creates a lot of practical ambiguity if some people revert to answering on a 0-1 scale, since having all one's answers be less than 1% is not unreasonable.
I object to the omission of "Anarchist" as a political orientation. Left-anarchism is very much not the same as Libertarianism, and it's even more distant from any of the other options.
#21 does't load; the message is "A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?". I do allow cookies. I'm on MacOS using Safari, but at a guess this is the result of a bad URL rather than a browser issue.
For the mental health conditions in survey 0, I assume that if I've had a diagnosis for a condition before but wouldn't qualify for it anymore, that still counts as "I have a formal diagnosis for this condition"?
On 6: weight gain/loss, there's no question about having given birth some time before especially a gain, which is often relevant
I have a question about the demographic question about depression. If I both believe I have un-diagnosed depression and have a family history of diagnosed depression, which answer do I give?
The question is:
"Depression
Regarding clinical depression...
(a) I have a formal diagnosis of this condition
(b) I think I might have this condition, although I have never been formally diagnosed
(c) I have family members (within two generations) with this condition
(d) I don't have this condition and neither does anyone in my family"
b and c both apply. Some other demographic questions have this issue as well.
The questions on the Metaphysics survey are so hard to respond to that I gave up. For example, the question: "How certain are you that racism is incorrect?" Racism is an act, not a proposition! How am I supposed to answer that in a meaningful way? Many of the other questions were ambiguous or didn't have an option for the answer I would actually give.
I think I actually did complete all of the ones that applied to me. What can I say, slow day at work.
I wish #4 had at least one free form text input field, and I suspect my answers will appear inconsistent to its author.
To summarize what I would have said: I know a lot about how most digital technology works internally, but experience the user interfaces as having been designed to be hard to use, hard to learn, and most likely to intentionally exclude older users, as well as those with a number of disabilities.
I no longer consider myself "digitally literate", since I can barely perform basic functions with the latest cell phone style of interface (no text, lots of icons, UI elements invisible until you make the right gesture at some unmarked spot on the screen, no help, no user's guide)
I did all the ones that applied to me except for two, where the formatting was counterintuitive for me and I kept making mistakes, so I didn’t want to submit with errors
That was extremely fun, but I have some questions.
On the Metaphysics survey, it went from page 1 of 3 to page 3 of 3. I checked, going back and forth, and it came up the same. Is there (a) a missing page 2 (b) ha ha, deliberate error, this is a metaphysical question?
On the Weight/Gain loss survey, it gave a great selection of "is this why you lost/gained weight" but nothing as simple as "went on a diet" or even "Other - lost weight because of this thing, gained weight because of that thing" which I do think is a flaw. I lost (and gained back) weight over a two-year period, but it wasn't because of "broke up with a boyfriend" etc. reasons
On the Covid-19 Impact, I skipped the wages question because (a) I can't remember exactly what I earned back in 2019 (b) I certainly can't be bothered trying to work out "if I worked X hours per week at Y rate per hour, what is that over a year?" and (c) none of your business, toots: if you want to know did I lose money during the pandemic, just ask that!
Question: Does everyone have a race? It's a social construct after all. I'm sure the american society could possibly assign one for me, but I've never been there. Does it only go by apperance? Then why would we need another word for it? It also seems different from the concept of 'people whose ancestors came from the same place look somewhat similar'.
I know about the falsified scientific theories about different human races, but you probably don't want such bogus categories in this neat little dataset.
I don't really want to describe where my family and I come from and live, because then people might try to invent a race for me instead of answering the question ;)
My mom's puerto rican and my dad's a borderer how do I answer race?
Survey 23, "Metaphysics" leaks the survey results after submission, meaning you can just go and look at everyone's answers, including their identifier. https://i.imgur.com/Gdo8Sa3.png
This seems like a bad thing.
My quibble on the demographics survey: it treats "having a mental illness" and "not having a mental illness" as a binary. What if I was diagnosed with the condition in the past, but no longer have it?
Completed all that apply to me, except 21 which is broken as others have reported already. Officially bragging.
"Metaphysics" was very fun to do, although the two math problems seem exactly equally hard to me.
"Understand your own beliefs" was also nice because I was surprised to actually feel represented by some answers. Spotted some conflict-vs-mistake-theory in there :)
The one about weight gain / loss left me somewhat confused. Not much happened directly before my birthday in 2018, and I had no idea why it was relevant.
"Health-Related Quality Of Life" was not explained very well (too wordy and therefore more confusing that it needs to be), but the research topic is super interesting and I hope something relevant comes out of it. I had to concentrate a lot while answering, though.
Can't load 21. Error: "A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"
Survey 9 doesn’t work. After submitting a few pages of responses, one page doesn’t let me submit, instead telling me “ The question has changed. Please review your response.” This happened several times.
I took other surveys and they worked fine.
(ROT13'ed; please don't read unless you have completed or don't plan to do the digital literacy survey.)
Ner Cebklcbq naq Svgvoyl va gur qvtvgny yvgrenpl fheirl vagragvbanyyl zrnavatyrff grezf gb grfg jurgure gur ernqre vf nafjrevat ubarfgyl?
For #0 on the psychiatric conditions I wish there was an option for 'I have been formally diagnosed, but have since recovered"
Another vote for Survey #21 not working - Chrome on Windows.
"A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"
Yes, cookies are allowed in my browser. No, I was not expecting a study result prior to submitting some information to the survey.
Chiming in yet again to say that “country that you most identify with” means nothing, and that you should split it into “country that you currently live in” and “country where your primary cultural background is from”, or something similar. I will continue to complain about this on every survey that has this question.
Also seconding everyone who complains about the race question. I already hate the US system of racial divisions, but if you’re going to use it, the bare minimum should be to allow for multiple selections and to have a place where you can write-in an “other” option.
ok enough griping time to fill in two dozen surveys
How am I supposed to procrastinate on this if you don't give us a deadline?
1. What should atheist Jews put for the religion question?
2. If you think I might have a psychiatric condition (no diagnosis) and also have family members with the condition, which should I check?
>Automation and technology is going to rapidly take away many people's jobs.
I always hate just answering with either agree/disagree on questions like this. I feel like most people's interpretation if I agree will be that it's bad and might generate a headline like "85% of economist are worried that automation is going to take away people's jobs."
How I want to answer that question is "Yes they will and that's a good thing. Here's why...."
Re customer satisfaction survey --- I read ACX via the RSS feed. So if the email letter stopped I would keep reading it but I wouldn't go to the website.
On the demographic survey, it should probably ask how long people have been reading ACX, not SSC - or maybe mention both. But some new ACX readers might not even know what SSC is.
I guess maybe I should have skipped the psychadelic survey entirely, but it was weird that there's no way to answer "I tried one and nothing happened".
I frequently find myself confused by the questions in the "Metaphysics" survey — sometimes none of the possible answers seem correct at all, or else the question is based on a false premise. Oh well; guess I'll just try to pick the least incorrect option, and skip the worst offenders. Hopefully the results aren't announced without describing what the options were.
I can't tell if "how long have you been reading ACX?" Is supposed to be a trick question. The first post was (just) less than a year ago, but I've been reading Scott's writing for at least a decade, so I assume I'm supposed to answer more than 2 years?
The weight gain/loss survey does not ask about pregnancy, and there was no comment section. This seems like a significant oversight.
I did them all. But I didn't read the first survey clearly and failed to get a proper user ID for half of them. Do I still get the praise?
When I try to load 21, I get: "A problem occurred:
Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"
> When you dream, does it occasionally happen that the *you* is a different person from your waking self? If so, is that person male or female?
I'd say in the last 5 years I've had a dream that I remembered upon waking 10 times, and even at that never to the level of specificity that would allow me to answer these questions.
I answered everything I could and it was fun! A few comments:
- Americans (as in the USA, but "people from the USA" is a mouthful) obsession with race is downright bizarre coming from a European point of view. Maybe a survey about that would be interesting?
- I've seen the word "community" used and thankfully it was explained (a city, town). I often hear it, and feel like it's also a very American point of view. For me, a community would be people you are close to, share something with. Family and friends mostly, maybe a few colleagues? But the question seems to imply that it could be two million people. Does that word has a strict meaning?
- Behavior during COVID pandemic: some questions were necessary to answer while a question before essentially made them moot, so I had to reply with bad data.
- Weight gain/loss: no mention of the COVID pandemic? That seems like a big oversight.
- 1 to 5/1 to 7/strongly disagree to strongly agree questions in general: I'd like a metasurvey about this. I have a hard time determining if I feel more like a 5 or a 6 on most issues. Does "strongly disagree" means "equivalent to the thing with you disagree the most", or "equivalent with the thing that you saw people disagree the most"? For example, I think most people are more vindicative about issues than me. Does that mean that I should only answer in a reduced scale, or should I use the full scale. Are there studies in this area to determine if this kind of question is meaningful even if it's only relative? Is it a case of the perfect being an enemy of the good?
- again a metasurvey proposition: I might be the exception in this community, but I don't feel strongly about most moral issues. Do we have any data on that?
- Understanding your beliefs: Is it bad to be a homosexual: I don't know if it's on purpose, but it's confusing to answer. I don't have anything against homosexuals in general, but I've endured some suffering because I was one that non-homosexuals haven't endured.
On the Covid-19 Impact survey, for the question "During the COVID-19 pandemic, did your employment require in-person interaction with the general public?", it doesn't give the option to indicate that it's not applicable because I was not employed. I selected "unsure / prefer not to say", but I see that the survey creator is active in this comment section, so perhaps they can give me some guidance.
My answers to some of the sex questions are completely dependent upon how in on it the other person is. Like, touching a stranger at a sex club where everyone there is like "yeah, this is where we will get touched by strangers" is completely different from touching a stranger on the bus.
I literally don't fall into any of the political buckets. Normally I'd say this is a matter of "I'm so special" syndrome but when you can't agree with at least half of any of the combined characteristics, what are you to do?
Small point, but in Survey 0, I interpreted the question "How long have you been reading ACX?" to include SSC as well. I assume this is what was meant, but some people might take it literally.
I'm sure you're inundated right now with people asking questions about the survey, but are the Life/Job/Social/et-al. satisfaction sections gone? Those results always seemed the most interesting to me.
Like, lawyers reporting higher average life satisfaction than software engineers was an unexpected and interesting result that kind of cut against the voguish archetypes of the unhappy, stressed lawyer and the chill, relaxed software engineer.
If they are gone, maybe they can make a comeback in future iterations?
Completed all of them, except for Optical Illusions, which did not load in the time it took to take all other surveys.
Filled out most of these surveys, but the only one that caught my interest was survey 23, metaphysics. I would love to see the data on correlations between answers to those questions, but there is no information of who the survey is made by, or what the data will be used for.
22 More Psychedelics the questions about health insurance are less meaningful if you live in a country where that's not the main way people get health care. (As is the case for much of Europe). I answered as about whether whoever the healthcare provider was should provide it which I assume was the intent
I just finished all except 2 that didn't apply to me and the Optical Illusion that doesn't work.
In "behaviour during covid" the question "If any of the following influenced your perception that the pandemic was easing up or ending, please check all that apply: *" seems to list a bunch of things about my personal experience and feelings. Which might effect how much I follow the lockdown rules, but are a separate thing from whether I feel it is easing up or ending
I did 23 of them. Didn't do the optical illusions because I got the same error as many below. Probably not how I should have spent my day.
In the "can society be wrong about words" question on the metaphysics survey, the question sheet only showed me two options but the survey results page shows four. Also, that was the only survey (that I took) which had a "see previous results" link at all after submitting – was that intentional?
why does the weight survey not ask about having children?
Regarding #14, businesses being "larger than some countries" isn't saying much in a world containing Vatican City and Andorra and San Marino. Yes, I'm totally fine with a company being larger than Andorra.
#6 let me submit a response even though I didn't click an answer for the second question... oops.
The rhyming survey is labeled as "targeted at non-native English speakers", but the survey doesn't actually seem written with that assumption. "English" is an option for native language, and "US English" is given as an example on the dialect question. It looks more like "we're studying how non-native speakers differ from native ones" which means that responses from native speakers are needed in order to have a control group. The label may be discouraging some native speakers from usefully participating.
At the end of the cryonics survey it says "Please check reddit.com/r/cryonics in January 2020 for survey response data. " So unless the cryonicists are branching out into time travel I think there's a typo
Some surveys don't ask for the ACX ID; mentioning in case this is not intentional. (Sorry, did not keep track which, was focussed on going through all of the surveys.) I added it in a freeform comment field for one of them, but that wasn't always possible.
I just reused the SSC ID I gave myself in a previous reader survey a few years ago, by the way. I'd saved it in my password manager and there was no reason not to reuse it, really.
Thanks for the surveys, everyone who made them! :)
Scott: It wasn't clear to me how to record a historical rather than a current diagnosis in the mental health questions in the demographic survey. I can see in the comments other people had the same issue. Maybe change the wording to "I currently have or have had a diagnosis" next year?
Other survey writers: please understand that not everybody is from the US, and phrasing your questions assuming everybody is American makes them hard for us to answer, affects your data quality, and also drives many of us slightly insane. For example, if you want to know my opinion on right wing people, that isn't the same as my opinion on American Republicans.
Unless I missed it, incest survey did not ask for unique ID.
For #2, i assume i should exclude stepbrothers and half-brothers with a shared father?
The Optical Illusion survey doesn't load for me (gives me an error about not being able to find the study result Id).
Did 'em all other than the non-functioning optical illusion one and those (if any) that were demonstrably intended only for demographics excluding my own. The metaphysics one was repeatedly infuriating, demanding that I choose between two options neither of which I could honestly endorse.
Survey #3: What do the colors mean? There's red, yellow, green, and blue, which are supposed to represent some of different health status, but none of those are mentioned in the survey text.
Scott, there was no place to insert my random number identity in the Incest survey.
I never understood why people fill in nonsense on surveys, until I got to the option "An unrelated man and woman: sexual relations are illegal."
So tempting...!
I found the "Personality" questions extremely frustrating - was it intentional that many (most!) of the questions had contradictory descriptions?
I see "humble" and "unselfish" as entirely separate and unrelated traits.
Same for "Interested in art, deep", "Amoral, carefree", "Ethical, unforgiving", "Principled, uptight", and "Imaginative, odd".
A couple of those pairs are close to opposites.
Question about how to answer the "work status" question on the reader survey: There is an "Academics (teaching side)" But there is no "Academics (research) side"
I work in a university lab doing research. I do literally zero teaching.
My best guess is either non-profit, or "governmental work" since all of our funding is state or federal grants.
Dos anorexia proper - not anorexia nervosa, but "accidentally loses weight due to insufficient appetite" - count as an eating disorder?
The "neither does anyone in my family" for the mental health questions is ambiguous. Do you mean genetic family (ie you're interested in the genetic connection & no one else counts) or family in a broader sense (in-laws can give you social experience with people who have the condition but won't pass it on genetically).
As a point of self-comparison, I seem to be getting half a point more politically right-wing each survey. I wonder if this'll keep going, and (if it does) if I'll move far enough to actually vote for a right-wing party by the time I'm actually able to vote somewhere.
I was completely unable to load the Optical Illusion survey. Error message "couldn't find cookie result for study result ID 31188. Are cookies allowed in your browser?" Yes, yes they are. Am I trapped in a metaphysical anomaly?
Regarding the Survey on feelings of gender:
"To what extent do you find yourself behaving differently when in a group of all/mostly women compared to when in a group of all/mostly men (which are otherwise equivalent to you)?"
I don't know, I haven't been in such a group in aeons, unless you count the internet, which makes gender harder to track.
"You wake up one morning with your body changed dramatically in one way, but otherwise as healthy as before. Which bodily change would feel the strangest/most disruptive to your perception of self?
30 cm taller than you were before
With the opposite sex organs (male to female and vice versa)
With a drastically different skin color (e.g. black to white and vice versa)"
Is this about swapping primary sex characteristics only, such that, say, a cisgender person would wake up with a body comparable to a transgender person of their gender who was unusually lucky with HRT results but had no surgical interventions, or is this such that a cisgender person would wake up with a body comparable to a cisgender person of the opposite sex? I am tentatively assuming the latter, but the phrasing is ambiguous.
Finished all the surveys that pertain to me, other than 3, as it was asking me to imagine things I cannot imagine, and to evaluate them in a framework which is utterly nonsensical to me. Answering surveys can be quite fun, it seems!
Since 31 is a redirect to 19, if we finish 31, should we do 1 or 20?
I think the "intellect significantly enhanced" option in 3, while a decent specification for an ordinary sample, was not a great choice for ACX; it specified as IQ 140-150 and IIRC the average IQ here is nearly that, so it's going to return a whole pile of "no use" and "negative use" results.
Prediction (80%|somebody runs the numbers): number of surveys completed will be higher in people who reported being depressed.
Half way through "Attentional Focus and Performance Orientation" it puts "I agree a lot" on the left, with the lower numbers, and "I DISagree a lot" on the right with the higher numbers. This is opposite to basically all the other surveys and most surveys I've taken. It was jarring when I noticed, had already answered one question without noticing. I expect many people will answer those questions incorrectly. I suggest switching it around ASAP.
"Scandinavian countries: heavily-regulated market economy" - this seems like a gross over-simplification, one that shows up in annoying political debates and makes them more annoying.
Optical illusion survey worked for me and my eyes hurt and I want to slap whoever came up with it. It went on *forever* and it didn't even tell me how many more I'd have to do before I could stop without it chucking my data!
The incest survey is hard to do because it's hard to let go of doubts regarding consent and power dynamics and assume everyone is a fully independent and consenting adult as the survey asks you to do.
Okay, I got through most of them, skipped a few that weren't aimed at me or were very difficult to understand. I will accept your praise slash concern.
The personality survey bundles together things that I feel shouldn't be bundled together at all. For example, I do feel "down to earth" but not unimaginative. I'm ethical, but not unforgiving. I answered natural on these question though cause if they were separated i would give opposite answers.
Does the Optical Illusion one record my results if I stop partway? I eventually gave up and quit. I'm not even convinced that it was ever going to stop.
I'm surprised you didn't add Georgist to the list of Political Affiliations after the book review contest winner.
I think it can discourage many people that the survey is hosted by Google Forms. Even though you do not type in a name, Google has numerous way of tying the answers to your identity and will have them stored for god-knows-how-long. Given the personal nature of some questions, it seems to have potential for abuse. Given how trivial it is to self-host a simple survey form, it would be the respectful thing to do w.r.t. your users.
I had to give up on #2. I have no brothers.
21 is a very unpleasant experience and takes too long - the visual illusions made my head dizzy and my vision blurry. I tried keeping at it for as long as I could, but I just couldn't finish it.
You should change "Alt-right" to "Nationalism" next time around, what you're describing as alt-right is simply nationalism.
Do you strictly want this to be a reader-only survey? What if this blogpost is someone's first ACX read - would you consider them as being an 'ACX Reader'? I want to let readers of my newsletter know about this survey but I do not want to adulterate your survey with non-readers. Let me know.
I don't know what to enter in the "work status" question. I work for a UK University. Like almost all UK universities, it is an independent but largely publically funded institution. It is technically a charity. So it's not for-profit, nor technically government, but it's not normally perceived as non-profit.
In the UK, we'd normally describe universities as "public sector", so I'm guessing that translates to "government" in USA-speak, even though we're not actually part of the government.
Regarding the ACX survey (24), going through it clarified my thoughts about the newer blog and Scott. I would definitely pay for a published compilation of the old essays & fiction, but I don’t read this blog with the intensity I read SSC.
Nowadays it’s a casual read. I like to read professional writing that’s not in my field, and I read In the Pipeline for a similar reason. SSC essays had a style wit and presentation that I really liked.
General demographics "profession" subtype computers: need to choose either "practical: IT, programming" or "AI". I object to the claim that AI is either not "practical" or does not require "programming" - I do practical programming of ML systems, which is what almost everyone means by AI. Does the survey mean "AGI hard research"?
In survey #4, "Digital Literacy", it asks you whether you're familiar with a list of technical terms. One of these is "proxypod", which is presumably a fake term thrown in to check if I'm just clicking "yes" to everything - but it sounds to me like it *should* be a brand name for a low-power VPN server.
In survey #0, "General Demographic Information", it asks how many children you have. For men, in particular, it's possible to be uncertain of this, but the field requires a specific number rather than a range.
Hey, this is kind of a weird comment, and I don't want this to sound preachy or anything but I am a little disappointed in the categories when it comes to the profession tag.
I currently work at an Amazon warehouse as an associate. Not one of the full-blown fulfillment centers, but one of the smaller delivery stations. While I have a pretty firm way of how I would describe my job to others: "Warehouse Worker", there is no option like that here. Okay so I should check "other blue collar." Fair enough.
But what if I still had my old job as a Dollar Tree cashier? What would I have put then? It is neither white collar nor blue collar under traditional definitions, yet not only do a majority of the people I know work in the retail industry, according to the Bureau of Labor Statistics, it is still one largest industries in the country in terms of people it employs. Keep in mind that Walmart, Amazon, and Kroger are the largest private employers in America last time I checked, and it's no coincidence that all of them have large retail/grocery components.
And I understand that a majority of readers who read this blog are more academically/IT inclined and you would want to capture the diversity on that front as much as possible, but I think you lose something when you have three different categories for IT and only two broader "Other" categories for what a majority of working Americans actually do. There could be interesting differences between the retail workers and warehouse workers that now won't show up because of the broadness / ambiguity of the definition. I know I have a lot more back pain working at Amazon now than I ever did at Dollar Tree. You can't even write in an answer to clarify the "other" so it is a difference you may never know.
Again, sorry for the rambling, I've been a fan of this blog and its predecessor, but that element of survey has always rubbed me the wrong way. I understand that it may be too late to substantially alter it now, or there might be space limitations, or other good reasons to keep it the way it is that I haven't thought about. Whatever the case is, I just thought I would mention it.
Thank you for your time.
I answered three surveys: one assigned, one out of interest, and one about Covid that turned out to be pointless because I live in New Zealand.
All surveys complete! Except the optical illusion one which wanted a cookie; I see others have had the same problem.
How to make your ethnicity surveys Brit-friendly, by a Brit:
* "Asian". In North America that means East Asian unless you write in South Asian. In Britain it means South Asian unless you write in East Asian. So a South Asian Brit who clicks Asian will find themself categorised with the American East Asians instead of the write-in South Asians. I know you don't want 10 options plus a write-in box, but I'd consider splitting those out.
* "Hispanic". In Britain, Hispanic is a fancy word for a (typically white) person from España - c.f. Gallic, Teutonic. "Non-Iberian Hispanic" is a useful modifier, or just call it Latino (which also avoids the arguments about whether Brazilians are Hispanic).
* "White/European". Try to avoid this one. Millions of people of colour live in Europe and are proud Europeans, and with Brexit and the racial abuse of England's non-white footballers at the Euro Championships being hot topics, it's a bad time to equate not-White with not-European. "White/Caucasian" is better; "White" is perfectly good.
Unrelated to race: the average Brit doesn't know what an FDA is. I would hyperlink FDA and put "or your country's equivalent".
The average Brit also doesn't have interesting opinions on American politicians or the state of American discourse. Can you let us know whether to answer for Britain or to skip those questions?
One of the surveys had Agree on the left and Disagree on the right such that "1" was strongly agree and "9" was strongly disagree. I wonder how many people are going to have read the words above the numbers?
Also the metaphysics survey was way to hard for 1:30am
JohnWittle, your astralcodexten user ID is s7wvK3jg9XFHpqqX
just google 'site:astralcodexten.substack.com johnwittle survey user id' to find this comment next time you need it
The "Family Demographics And Homosexuality" survey is confusing because it only asks for ASAB, not gender, and then it asks "Are you exclusively homosexual?". I'm not sure what is meant by "homosexual" in such a context.
Right. Are half-brothers to be included? Same treatment for 'via father' and 'via mother'?
#13 is for psychedelics users only. Is #22 for everbody?
So I did a bunch of these yesterday. Early this morning I have this long evolving dream. I’ve taken a new job and my new employer is administering these these endless seemingly pointless multiple choice tests to see where I’ll best fit in with their teams.
Oy. This is never going to replace my favorite. I’m the captain of a pirate ship, straddling a canon. My crew is the University of Minnesota women’s volleyball team. I can’t help but think that one means something…
From 'Much less than normal' to 'Much more than normal' -- does that ask the personal 'normal' (healthy, i.e., 'Me before X'), personal average, or a population average?
Incest survey doesn't ask for user ID; intentional?
I had one of those awkward both/neither reactions to the incest survey.
I want a blanket prohibition on sibling incest. I don't want the legal code to have to get into the minutiae of which sex acts are illegal under which conditions; that's just an invitation to lawyers to find loopholes. I also think sibling incest is a *bad* choice, and don't want people advocating it any more than I want them advocating smoking.
But under the exact conditions specified in the survey, I would issue guidance to the DA/CPS that it isn't in the public interest to prosecute anyone. So the "restrictions" wouldn't actually restrict anything.
It would have been nice to be able to give direct feedback about a survey to the survey author. I realize I can comment on it here, but don't know if that will get through. For example, after answering several questions on the "Health Related Quality of Life" I felt the comparisons weren't specific enough for me to give accurate answers (so I decided to bail out rather than skewing things)
In 21 (optical illusion), I see the red cross and then it all goes black, which seems to me like unintender behaviour. I'm using Brave browser. I have already answered the first two pages of questions, so it's annoying to test with another browser.
I said I was "hispanic" given the options, but that didn't feel like a "race" question to me, but nationality. The alternative being "white (not hispianic) [where hispianic is how americans call people from certain places]".
Why not set up a site to do the randomization and redirects automatically?
I found the 'Personality' survey very frustrating to take. Most of the questions seemed ambiguous at best. Do they want us to answer based on BOTH words, or based on EITHER word? For instance, 'active' and 'talkative'. Do they want a response to 'active AND talkative' or a response to 'active OR talkative'? Most all of the questions suffer from this defect. What if I'm talkative but not active? What if I'm carefree but not amoral? The author apparently feels that words like 'emotional' and 'anxious' are synonyms or something. Not true.
I quit 17 minutes into the Optical Illusion test that supposedly takes 10-15 minutes. I hope it at least saves the data part way.
That test *seriously* needs to be a) much shorter and b) have a progress bar. After a while, I started to suspect that it was secretly a test to see how long keep would keep doing the same thing over and over before giving up.
"Understanding your beliefs" had several US-specific questions that took me fairly flat-footed. I live in Canada.
I completed every survey except that fucking optical illusion survey. It said 10-15 minutes but it felt like an eternity.
I am probably skipping the survey this time because of the trivial inconveniences that exceed my threshold. Instead of filling one long form, it's "create a weird id, copy paste it, keep it some place for every other form, insert it into the next form..."
I wish it was more like "this cookie will be used as your default id for the survey, you can edit/replace/delete it as you see fit, should you care enough", then each form would end with a link to the next one, id-preserving.
The ACX Customer Satisfaction Survey made me think about why I considered becoming a paid subscriber to ACX, but chose not to (so far). I certainly enjoy ACX, am a paid subscriber of other things, and could easily choose to subscribe to ACX.
After thinking about it, I'd say the main reason is actually Scott's repeated downplaying of the importance of subscribing, and of its benefits. It's basically always presented as *super* optional, with an underlying message of "don't feel bad if you can't subscribe, it's really not that important, all of the good articles are free anyway".
I understand why it's presented that way, to avoid some people's scrupulosity issues, but it's honestly so much of a downsell that it makes paid-subscribing sound like an unattractive option. It sounds like "if you *really* want to throw away your money, then be my guest, I guess".
I am sure this was discussed to death already, but I lost the memo. What does Hispanic mean and how is that in the race category? I am Argentinian and so I don't identify as white (mainly because all white countries have decided we are not white, so I follow suite here and do not want to identify as white) but neither do I identify as "hispanic". In the actual hispanic language, that means Spaniard, which I am not. In the unitedstatezen language, as far as I know, hispanic means "Mexican looking", which does not apply to me either, neither by personal identification nor, I believe, by the standards of the natives that came up with that word (I've had 4 Unitedstatetenzens, as well as 7 Europeans and dozens of Chinese, tell me straight to my face: "ohh, so you are Argentinian but also obviously (never understood why is that obvious) not Latino/ not Hispanic, what race are you?"
I chose White, at the end. Does that mean that all my survey answers will be read as belonging to some anglosaxon protestan puritan or similar? Probably not very important, but in my mind that would disqualify the results at least a little bit.
(I really would like to know what Latino/Hispanic is suppossed to mean and why I so obviously do not belong in that to the people that came up with those words. Actual Latinos/Hispanics do not use those words nor do we feel we are a "race")
Just here to boast - did all of the ones I was demographically qualified for... except the optical illusions one, I couldn't get that one to work (chromium, Linux).
I've been reading OB/LW from way before LW even became a thing. I feel like a large part of the impact that cryonics has made on rationalists stems from its discussion there. My life doesn't revolve around cryonics, but I'm sure I've read most, if not all, of the posts about cryonics on OB and LW. **I can't recall ever hearing the term "biostasis"!** Using Google's site search, there doesn't seem to be any mentions of the term on LessWrong until this year...
Someone tell Scott that there needs to be a "got pregnant and then gave birth" option on the weight gain/loss survey. Pregnancy and childbirth are the only reason I've ever gained or lost weight, but my weight history looks completely bizarre without noting "gave birth then."
I don't understand why pregnancy wasn't mentioned/asked about in the survey since it is probably the single biggest reason why people suddenly gain and then lose more than 20 lbs in a short amount of time.
Survery #4 is rendering oddly. "of trying to gureit out myself." it has an up arrow instead of "Figure it"
I completed all of the non-targeted surveys except 7,(which may merit a "targeted at people who have ever meditated note, I quickly grew bored with skimming through long lists of things that read as nonsense to me just to check the "no" button), and 21. (Took way too long and felt like no actual end was in sight, beginning to agree with the suspicions posed by other commentators that the whole thing was an exercise in testing patience.)
Of the set, I thought 16 was the most interesting. I'm looking forward to seeing investigations of those results, and what the intent of the survey was.
Just ran the table!
Survey 5 "targeted at people with depression/anxiety" -- The questions all relate to symptoms and treatments for depression. It instructs a person with anxiety to word-substitute anxiety where it says depression throughout the survey.
I don't think doing that is going to yield meaningful data about people with anxiety. Anxiety and depression not uncommonly co-occur, but they are different things with different symptoms, different coping behaviors, and different treatments.
i'm not sure if it was there and i just missed it, but i don't think the meditation survey asked about *when* you experienced the weird religious-type experiences it's asking about, or the context in which they happened? i'd venture to guess people with, for example, charismatic christian backgrounds or schizotypy would have plenty of experience with weird religious-type experiences.
The weight loss survey included questions about life events that looked like they were trying to approximate something like the Holmes and Rahe stress scale, but it was missing the most impactful questions (e.g. "death of a spouse", "divorce", or "imprisonment"). An oversight, or did the survey had other things in mind?
Survey 8: “Almost all politicians are liars”
Is there a difference between lying and deception? The former seems more specific to me.
Nitpicking, but France's National Front doesn't exist anymore (survey #0). It has been replaced by the "National Rally".
So i gave up on the optical illusion one because it seems to go on forever with very similar or identical optical illusions. Did anyone actually complete it?
I ran into on the first trial of optical illusions after the practice trials. In case this helps debug: "Error in Fixation Cross (eventAPI.js:101): during event after:end : EncodingError: the source image cannot be decoded."
What with the amount of complaining here, I haven't even tried out the surveys. Are there any surveys people think are well-designed?
I am having trouble with the COVID questions. These seems to be no option for "I absolutely did not care at all". I have never been even the slightest bit concerned that I would "get" COVID or be harmed in any way by it if I did. Biologically, it doesn't even exist as far as I am concerned. So questions like
"During the Covid pandemic, which worried you more (choose 1):"
Are meaningless.
Did you model these after the tautological and undecidable "COVID questionarres" that have to be filled out all the time for insurance coverage?
-- "In the last 14 days, have you possibly been in contact with a person who may have been exposed to COVID?"
Uhhh...how can I possibly answer that except with "Of course, that is possible"?
I feel I may have to punt this questionarre because I can't answer accurately.
__________________________________
Let me go further, because I feel like my perspective is unique and might have some value to the community, which is why I hope it can be capured in the survey(s)
[REDACTED FOR BREVITY]
BRetty
This survey is so much worse than previous surveys that I failed to do more than the first two and gave up halfway through 3. I put this down to the hassle of the user ID, having each part separate without indicate who should fill them in (2 is only aimed at men),the awfulness of the questions in 3 (as a disabled person they made me feel physically sick) and the overall lack of sensitivity (how hard would it be to include a non-binary option for instance). I've lost my user ID now so I won't be doing more but I though you might like to know, as the survey was never this bad before
Regarding #19 (Attentional Focus And Performance Orientation):
The second half of the survey gives a 1-5 scale with 1 labeled as "agree a lot" and 5 labeled as "disagree a lot". This is the reverse of what I would normally expect, and I almost didn't notice that higher numbers didn't indicate stronger agreement. I answered according to the provided labels, but other people might not notice the change or might assume it was a mistake and answer the reverse.
The question about health reminds me of https://xkcd.com/883/ -- it is easy to remember all things that are wrong or suboptimal with my body, but when I start to think about all the problems I *don't* have, or just list all organs in human body I know about and then consider how many of them never gave me a reason to complain, suddenly I feel quite healthy.
Now sure why should my weight be correlated to what I did in 2018, but whatever.
Note to survey authors: if you make a scale with both numeric values and verbal descriptions, please make "less" correspond to smaller numbers and "more" to larger numbers, otherwise it is a bit confusing. ("How much do you like kittens? 1 = a lot, 7 = just a little")
If your survey is long, please split it into pages of similar length and always say "page X of Y". It sucks when I am at page 10 and feel quite bored, and I have no idea whether there is 1 more page waiting for me (in which case I would rather complete the survey) or 20 more pages (in which case I would rather quit).
Is doing all the surveys (ignoring the ones aimed at people you aren't) meant to be some sort of great feat? It seemed easy and simple to me.
I so wanted to complete the quality of life survey, but found the instructions incomprehensible.
I'd be curious to find out what that "optical illusion" survey was *actually* testing for. Clearly, the "illusions" are supposed to distract us from something... but from what ?
Whoever wrote Metaphysics, your question about intersex conditions is *awful*. XXY/Klinefelter's is only "ambiguous between male and female" if you know absolutely nothing about sex chromosomes or their aneuploidies. I recognize this misconception is fairly common, but that doesn't make it not "so immediately dumb that I considered closing the tab instantly, because if all these questions were written with that understanding of their concepts then no meaningful results could possibly come from them".
FYI, Qualtrics surveys are not accessible over Tor, so on the more sensitive topics you may lose some people who would only answer from behind an anonymizing proxy.
On the other hand, I suspect a goodly percentage of the survey-takers could actually be uniquely identified by the information requested in the #0 demographics survey (though perhaps not actually located by it.)
For next year, it may be good to set a norm for Likert scale directions (i.e. either always have agree on left/disagree on right or vice versa). There's some variance across surveys and have realized a couple times that they were not set up the way I thought.
Wow, I had a really hard time understanding what was being asked of me in the hypothetical health states questions. I'm not sure how I would write clearer instructions for it, because I'm not entirely sure I get it still. The instructions almost certainly need to be shorter.
Nothing better demonstrates the questionable utility of surveys than filling one out!
I'm assuming #2 is for people with knowledge of their blood relations.
For me, the optical illusion test took a little over 30 minutes, plus breaks for eyestrain. Regardless of what some commenters suggest, there IS an end to it, it's not testing just how willing you are to keep doing a repetitive task. However, it does save the data even if you don't have the time to complete it.
I wish that the optical illusions survey (#21) had a button for "faded in completely, but I can still see/feel the rectangle border, so not an uniform image". Because in that case, you always have to wait while staring at a dot, and it requires a LOT of willpower. If I could just press A/B or something, that would make it a lot easier.
Also, I accidentally noticed that the rectangle border becomes more apparent when you slightly move your head while still keeping your eyes fixed on the red cross. I tried not to do it and be "good".
(this is a test to see what my username is)
The quality of life survey was nuts.
One, quite wordy and convoluted. Two, actually bloody hard! Had to think quite a lot about what appropriate answers might look like. Three, my priorities seemed to differ wildly from the author's.
Really looking forward to reading the results, the goal was one of those Basilisk extremes, monster cyborgs that no normal human or society would ever sign up for (hopefully), but there's more insight in it beyond the original idea.
Well, I spent nearly four hours on Metaphysics, and then after submitting, reopened the test to see new questions about... fan conventions.
I shan't be answering those!
I wonder how many other outliers are like me, and simply high school graduates, and what they're like. Whether they're all grumpy and cynical like life beat into me, or if I'm just an outlier among outliers. Or maybe they all have other mental illnesses like ADHD or other heritable disorders. It's really difficult to find outlier-based surveys where I might glean some insights into "my demographic", as generally "high school and some college" results are lumped together into "poor and ignorant" instead of "chronically curious but unable to pursue academia". Will you be doing a separate analysis of such persons? Are there even enough of them? Please do, I would like to know just to see if my psychology correlates to this or other factors.
I started two surveys and had to quit both. My (undiagnosed) ASD doesn't allow answering questions where I know my answer will be interpreted incorrectly.
For whomever is responsible for survey #3, you wrote at the bottom of the survey:
> It would be very helpful if someone could suggest a way I could randomly generate hypothetical health states in an online survey (i.e. randomly complete the five possible states in their three possible levels). I understand enough 101-level coding to write a program to do this on my local machine, but I don't know how / where I could host it in a way that would be suitable for this sort of survey.
Basically, I'd generally suggest creating a web application and hosting it on (deploying it to) a VPS – a virtual private server. There are some other possibilities for what's effectively free hosting too, and you _could_ provide public Internet access to a program running on your own computer (and for free), but a VPS is the most general way to host a program publicly. (But keep in mind that this can be arbitrarily hard – the Internet is very much a 'densely populated ecosystem' and a publicly accessible server will be 'under attack' almost immediately after accepting incoming public Internet traffic. I wouldn't overly worry if it's only going to be live for a relatively short period tho, e.g. a few weeks, or even a month or two.)
I'm open to answering any other related questions here too.
The metaphysics survey having "option 2" and "option 1" in that order is really aggravating. It might be random don't make me guess how to enter a right answer.
In the optical illusion survey, I often found that the images looked uniform *before* I was allowed to press the space bar. I fear that this design choice (to only allow perceived uniformity after a certain time) may significantly impact your results.
I just finished all of them except #21.
To beat the dead horse I mentioned in a previous comment on this post, I found myself, SO many times, wishing for another option/answer to a survey question, or even just an 'other' option/answer with a text box, for at least _some_ way to escape the provided 'framing' and challenge the assumptions seemingly baked into to the questions! Having now just been subjected to that, over and over too, I hope I can remember that feeling every time I ever read or hear about any survey results ever again.
Skipping #6 as I don't quite see how to answer it if I am not regularly measuring my weight.
#21 takes me to the depression questionnaire.
Rough showing for Christian Democrat political views. It was one of the dominant political parties and ideologies for multiple European countries throughout the 20th century. It is still in power in many countries today - Merkel is a Christian Democrat, for goodness sake.
But for this survey, that viewpoint does not exist. You can be a U.S. Democrat, or a Scandinavian socialist, but a Christian Democrat - nope, not on the table.
This is perhaps reflective of the readership and survey makers, but is not progress for the world in my view.
I was 80% through the autogynephilia survey when I clicked on "Previous" to check something in a previous page and was suddenly back at the first page with all fields empty. Was my data never saved at all and I'd have to start from the beginning, or was my partial answer saved and I can just enter my id again, maybe put this in the first other field, and continue where I left?
Woot! I took all the surveys that were relevant to me. Please feel concern for me!
An interesting thing to measure would be, for each user ID, how many questions they answered: there might be some correlations between that and other things.
+1 for "I tried the optical illusion, gave up after a while".
Pity, I am really into optical illusions, but having no way to "stop now" or knowing how long it last, no thank you.
Having them one per page is also pretty bad UX...
i took all the relevant surveys please praise me
On my phone, for questions like the rhyming survey, the higher values are hidden - I see only options 0, 1, 2 in the rhyming survey, I need to be aware that I can scroll to higher values, or all answers will end up 2 or less. Maybe, something to control for?
*Redoing the survey*
Can't give the Curium rhyme a 5 rating in the rhyming survey. Maybe an issue on my mobile phone.
Seems to work for all other lines
The Survey #19 is a bit weird with how the responses are presented. For the first part of the survey "often" is on the right and "never" is on the left. So the more that statement applies to you the more you select your answer on the right.
After about half the survey it changes and "I agree with this" is on the left and "I disagree" is on the right.
I had to correct a bunch of answers because I caught it in time but I could imagine that some people answer the second part of the survey with the opposite to what they actually mean.
I took the 19 non-targeted surveys, 1 targeted survey that I was the target of, and accidentally 1 targeted survey I was not the target of by going in order and not noticing it was targeted/not reading the parenthetical after the link. Oops. There's at least 1 incorrectly targeted response to the Rhyme survey.