Finally, just because I included an LLM summary of the comment history in the public modlog, it does not logically follow that the ban occurred because of the LLM summary. It simply happened to correspond very well to my own manual review
yeah, it’s ok because the LLM wasn’t hooked up directly to the ban API, you just used it systemically to not do the only fucking thing you’re supposed to be doing as a moderator
this human in loop shit is how corporations absolve themselves of responsibility for decisions taken purely on the word of an LLM. it lets them fire a worker instead of an executive. you’re sure this is the route you want to go?
yeah, it’s ok because the LLM wasn’t hooked up directly to the ban API, you just used it systemically to not do the only fucking thing you’re supposed to be doing as a moderator
Can you fucking read? What “systematically” are you talking about? How did we use it " only fucking thing you’re supposed to be doing as a moderator"? Is summarizing “the thing you’re supposed to be doing as a moderator”? Is doing a summary once “systematically”? Why are you continuing to spread disinfo?
Once again. The admin in question DID NOT USE THE LLM TO DECIDE ON THE ADMIN ACTION. Can you understand this? Can you read this? Am I talking to a wall?!
Once again. The admin in question DID NOT USE THE LLM TO DECIDE ON THE ADMIN ACTION. Can you understand this? Can you read this?
Literally no one believes this corpo speak bullshit. That they just coincidentally ran this unpublished python tool, did their own work, then just happened to use an LLM to do the exact same work right after, totally innocently? That reads as absolute ass covering and nothing more. This is the “I smelled weed” of cop stops, just filtered through nerdy fediverse bullshit.
Then, because the above totally happened like you said it did, as a one off joke that no one would ever notice, the same admin opted to put a current OpenAI model name in the LLM field in an absolutely not tongue in cheek way for other admins to totally catch and joke about? Which of course happened, haha, y’all had a big laugh about it before this blew up, yeah?
Oh, and of course this only happened the one time, and never again. Of course no one on your team used this unpublished time saving and thought terminating tool again, of course not.
Nothing has changed in that respect, except now we have an extra tool to make sure we get the decision correct.
And I thought the summary did a much more comprehensive justification of the ban than I could have fitted in the modlog, so why not include it?
idk maybe because it’s unnecessary to do that in the first place, and can be filled with nonsense from nonsense generator https://lemmy.dbzer0.com/comment/25880065 emphasis mine
you do realize all this comes across as running interference while feeling real guilty? you could just, like, drop a link to a repo with the bits. you know, with commit history. like i’ve been asking you for. just simply linking to that would be a real quick way to settle the issue! it’s even some free code review for you!
no but you’re about to be talking to a defed unless you stop this all caps fake outrage shit. should I do you the favor of deleting the garbage post you just made so you can give it another go?
this human in loop shit is how corporations absolve themselves of responsibility for decisions taken purely on the word of an LLM. it lets them fire a worker instead of an executive. you’re sure this is the route you want to go?
I completely agree with you. We have never and will never go that route.
Here’s the answer to the only question you posted which should be obvious from everything else I’ve said and done.
cool! please make it clear to the mod in question that they shouldn’t be using an LLM for anything in the future, even summarizing posts. make it part of your instance’s policies.
How could an admin possibly enforce that? What if a mod created a summary locally and never shared it with anyone? The ai summary wasn’t used as evidence, that is already policy and has been explained to you multiple times. You are shifting the goal posts to the moon, and no policy change will ever satisfy you.
assume I know and understand that the LLM did not literally do the banning
maybe there’s a large body of existing research on how even human in the loop systems confirm and worsen biases? maybe it’s a bit obvious when you go through the process the moderator took to get to their decisions in your head? slowly now, maybe you’ll get there
assume I know and understand that the LLM did not literally do the banning
I am telling, again, that the human did not use the LLM to think for them either. The admin took the decision to ban the user irrespective of the LLM, and the rest of our admin team and me specifically, would never let an admin become a “human in the loop”. The LLM was used just to summarize, as part of the test, with a misguided inside joke on using OpenAI tech.
I will readily admit that there was mistakes made by the admin. Not on their actions, but on their visuals. Because those visuals were spun to keep feeding this made-up controversy. We didn’t use the LLM, but it appeared like we did, and we already owned that.
The admin took the decision to ban the user irrespective of the LLM, and the rest of our admin team and me specifically, would never let an admin become a “human in the loop”. The LLM was used just to summarize
you don’t appear to have much understanding of how a human in the loop system works in practice. LLM summaries are used to confirm biases, especially when the prompt is something along the lines of “do these posts contain <negative term>?” though these systems are stochastic so you’re going to get unpredictable biases regardless of the prompt.
I don’t accept that the LLM summary didn’t influence the decision because the mod in question confirmed that he knew the LLM agreed with him (that’s bias, and also not something LLMs are capable of actually doing) and because if it didn’t, then the summary is worthless
which is why maybe you should just not have them in the future? just don’t touch LLMs when you’re doing mod work. either there’s no reason for it or you’re doing something monstrously wrong.
I don’t accept that the LLM summary didn’t influence the decision because the mod in question confirmed that he knew the LLM agreed with him (that’s bias, and also not something LLMs are capable of actually doing) and because if it didn’t, then the summary is worthless
In this case, according to the admin in question, the LLM summary came after the decision, as a sort of a test. I.e. the admin made a decision, and wanted to see if an LLM would subsequently agree with that decision. In this specific case, it did, which is why they misguidedly decided to keep its summary in the modlog (opening us up to this whole shitstorm), but ultimately, that admin anyway decided LLMs in the mix is not good at all, which is why you never again saw an LLM summary in the modlog.
Having a LLM confirm a decision is the same thing as having the LLM make a decision and then figure out if the mod agrees with it. If they would have chosen not to rule based on the LLM output, then the LLM was part of the decision making process. The order does not matter.
Including the LLM outputting something that implies a determination at any step automatically makes it part of the process.
yeah, it’s ok because the LLM wasn’t hooked up directly to the ban API, you just used it systemically to not do the only fucking thing you’re supposed to be doing as a moderator
this human in loop shit is how corporations absolve themselves of responsibility for decisions taken purely on the word of an LLM. it lets them fire a worker instead of an executive. you’re sure this is the route you want to go?
Can you fucking read? What “systematically” are you talking about? How did we use it " only fucking thing you’re supposed to be doing as a moderator"? Is summarizing “the thing you’re supposed to be doing as a moderator”? Is doing a summary once “systematically”? Why are you continuing to spread disinfo?
Once again. The admin in question DID NOT USE THE LLM TO DECIDE ON THE ADMIN ACTION. Can you understand this? Can you read this? Am I talking to a wall?!
Literally no one believes this corpo speak bullshit. That they just coincidentally ran this unpublished python tool, did their own work, then just happened to use an LLM to do the exact same work right after, totally innocently? That reads as absolute ass covering and nothing more. This is the “I smelled weed” of cop stops, just filtered through nerdy fediverse bullshit.
Then, because the above totally happened like you said it did, as a one off joke that no one would ever notice, the same admin opted to put a current OpenAI model name in the LLM field in an absolutely not tongue in cheek way for other admins to totally catch and joke about? Which of course happened, haha, y’all had a big laugh about it before this blew up, yeah?
Oh, and of course this only happened the one time, and never again. Of course no one on your team used this unpublished time saving and thought terminating tool again, of course not.
Come the fuck on.
idk maybe because it’s unnecessary to do that in the first place, and can be filled with nonsense from nonsense generator https://lemmy.dbzer0.com/comment/25880065 emphasis mine
and previously, https://lemmy.dbzer0.com/post/67894781/25769171
unless i’m to disbelieve my lying eyes, that does sound awfully like llms are in the decision making process
lol, lmao even
you do realize all this comes across as running interference while feeling real guilty? you could just, like, drop a link to a repo with the bits. you know, with commit history. like i’ve been asking you for. just simply linking to that would be a real quick way to settle the issue! it’s even some free code review for you!
You realize there might not be a repo, right?
you realize that mod probably shouldn’t be one, right?
As I said elsewhere, I’d rather this be a learning moment, than a purging moment.
the two things can go together, and not removing said mod means that if they do not learn they can still do plenty enough damage elsewhere later on
and I say this out of the position of having fucked up in a mod role before
seriously this just looks more and more embarassing the more you post about it
no but you’re about to be talking to a defed unless you stop this all caps fake outrage shit. should I do you the favor of deleting the garbage post you just made so you can give it another go?
think very carefully on this because I’m not sure pretending to not understand what I’m asking is working out for you
I completely agree with you. We have never and will never go that route.
Here’s the answer to the only question you posted which should be obvious from everything else I’ve said and done.
cool! please make it clear to the mod in question that they shouldn’t be using an LLM for anything in the future, even summarizing posts. make it part of your instance’s policies.
How could an admin possibly enforce that? What if a mod created a summary locally and never shared it with anyone? The ai summary wasn’t used as evidence, that is already policy and has been explained to you multiple times. You are shifting the goal posts to the moon, and no policy change will ever satisfy you.
oh no the goalposts! think of the theoretical shitheads who might do this in secret!
please see the pinned post and fuck off
assume I know and understand that the LLM did not literally do the banning
maybe there’s a large body of existing research on how even human in the loop systems confirm and worsen biases? maybe it’s a bit obvious when you go through the process the moderator took to get to their decisions in your head? slowly now, maybe you’ll get there
I am telling, again, that the human did not use the LLM to think for them either. The admin took the decision to ban the user irrespective of the LLM, and the rest of our admin team and me specifically, would never let an admin become a “human in the loop”. The LLM was used just to summarize, as part of the test, with a misguided inside joke on using OpenAI tech.
I will readily admit that there was mistakes made by the admin. Not on their actions, but on their visuals. Because those visuals were spun to keep feeding this made-up controversy. We didn’t use the LLM, but it appeared like we did, and we already owned that.
you don’t appear to have much understanding of how a human in the loop system works in practice. LLM summaries are used to confirm biases, especially when the prompt is something along the lines of “do these posts contain <negative term>?” though these systems are stochastic so you’re going to get unpredictable biases regardless of the prompt.
I don’t accept that the LLM summary didn’t influence the decision because the mod in question confirmed that he knew the LLM agreed with him (that’s bias, and also not something LLMs are capable of actually doing) and because if it didn’t, then the summary is worthless
which is why maybe you should just not have them in the future? just don’t touch LLMs when you’re doing mod work. either there’s no reason for it or you’re doing something monstrously wrong.
In this case, according to the admin in question, the LLM summary came after the decision, as a sort of a test. I.e. the admin made a decision, and wanted to see if an LLM would subsequently agree with that decision. In this specific case, it did, which is why they misguidedly decided to keep its summary in the modlog (opening us up to this whole shitstorm), but ultimately, that admin anyway decided LLMs in the mix is not good at all, which is why you never again saw an LLM summary in the modlog.
Having a LLM confirm a decision is the same thing as having the LLM make a decision and then figure out if the mod agrees with it. If they would have chosen not to rule based on the LLM output, then the LLM was part of the decision making process. The order does not matter.
Including the LLM outputting something that implies a determination at any step automatically makes it part of the process.