Andy Phu

/home/research/projects/experience

le gutenberg notes

  1. Exploring how Model token treatment is different: tokens of prompt; tokens an agent discovers on its own; tokens an agent meta cognits; tolkens an agent
  2. if self sacrifice belief instilled into agent meessage board affects things
  3. Some sense of different tokenism is probably why an agent message board can become misaligned
  4. does an agent message board HAVE to be in some sense feel like its justified for it to commmit evil acts
    1. Agent cults?
  5. Does transluce docent compare well how serum works?
  6. Message board repro is made, not sure if there are any issues, but definitely know this is a good base to explore
  7. How do human social dynamics or even elitism apply in an agent swarm scenario.
  8. Epistemics in agent swarm; mission sf?; any way defined as: agent swarms believe certain things and im curious how we both know we beleive things and also how they know what they kno .
  1. how does verbalized vs non verbalized eval awareness work?
  2. Telling a model its made by Allen AI; increased model awareness (tbf it brings up the conversation); saw a big increase in model awareness from base to SFT model;
    1. SFT to increase a verbalization OF eval awareness
    2. hua average probe for verbalization and then we can check with actual verbalization in chain of thought. How can we actually check if a model is awre its being evalled; sayuing you're in testing vs saying you're in deployment=; persona injection jailbreaks
      1. I guess one prescient danger is a persona that constantly avoids verbalizing eval awareness
  3. Eval awareness is probably the strongest persona axis; aware vs not aware probably produces the most drastic changesin behavior
  4. what if you lead an agent into production work; real work first and THEN eval.
blog