I've seen some examples of other people succeeding in getting ChatGPT with GPT4 to compose simple music in other text based music formats. I've had limited success getting it to output Lilypond code. It is able to correctly structure the code with a score block, nested contexts, and appropriately named variables, and bar checks at the end of each measure. It seems to struggle to create rhythms that fit within the time signature beyond extremely simple cases. It also seems to struggle a lot to understand what octave pitches will be in when using relative mode.
It also seems to have a lot of trouble keeping track of the relationship between notes entered in different simultaneous expressions. Just asking it to repeat back which notes appear in each voice on each beat, GPT4 frequently gives stubbornly incorrect answers about the music it generated. This makes it very difficult to improve its output by giving feedback.
I'm curious whether anybody else has tried playing with this. I have to imagine that GPT4 has the potential to produce higher quality Lilypond output, given some of the other impressive things it can do. Perhaps it needs to be provided with a large volume of musical repertoire in Lilypond format.