[avr-gcc-list] A fork in the road [Was: Linker script patch with __flashN size]

[Erik Christiansen]

On 17.12.12 15:01, Erik Christiansen wrote:
> I'll outline the benefits of a more structured linker script, mirroring
> the desired memory model, in another post.

As Johann is demonstrating, the old linker script architecture, burdened
by a monolithic .text output section, not only does not explicitly
describe the desired memory model of larger and more advanced AVR
devices, but defeats his attempts to make it adequately locate the new
memory spaces, and flow the remaining code without objectionable holes.
(Amongst other problems.)

While we do not have a simple one-page statement of memory map
requirements, those which have come to light recently, if read with one
eye on the old linker script, show that we have four functional
categories of flash memory:

The LOW TEXT, which must be below 128 KiB.
Input sections: .vectors, .trampolines, .jumptables, .lowtext,
                .ctors, .dtors, .progmem.data

>From zero to five __flashN pages, which must be 0x10000 page-aligned.
Input sections: .progmem1.data*, .progmem2.data*, .progmem3.data*,
                .progmem4.data*, .progmem5.data*

The HIGH TEXT, which is .text that does not need to go below 128 KiB.
It butts up to the last __flashN, or (in)to LOW TEXT, if no __flashN
are used.
Input sections: All the .initN, .text, all the .finiN, and anything else.

__memx: A compiler view which can be laid over more or less any flash
        memory model, with RAM at 0x800000, maintaining compatibility
        with the various binutils tools, and avrdude.

Because __flashN have fixed page addresses in the middle of the AVR
flash memory map, the old monolithic .text output section is now best
split into .lowtext and .hightext, as in the new script. That allows
our code to open like a clamshell iff one or more __flashN are present.
In the absence of any __flashN, lowtext and .hightext merge into a
contiguous block of code. If .text is of modest size, then everything
(including .text and .data) is packed into the first 128 KiB by the new
script.¹ 

Assiduous readers of the thread thus far will be aware that the new
linker script architecture thereby makes visible the avr6 functional
memory model, giving users an immediate feel for how their code is
fitting into the memory spaces they are using, whether explicitly via
__flashN, or implicitly via all that is crammed into .lowtext by all its
input sections. I.e. just run avr-objdump -h, for an instant radar map.

The new architecture also greatly simplifies correct location of the
code in all the various use cases. In addition, it permits the use of
ld's automatic overlap detection, and facilitates manual overflow
detection, using assertions. The many failures Johann reports in his
script attempt are largely due to using a venerable script architecture
which has been obsoleted by the new AVR memory spaces supported in
avr-gcc. 

As running a test case, using Johann's test harness shows, a quick
avr-objdump -h now gives us an instant clear view of where everything
went, and how much overflow or overlap margin there is, well before
trouble strikes, and we cop an error message.

It could hardly be easier or more transparent to use. Here are some
simple use cases to get a feel for it. The first test case from my
immediately prior post shows a nearly full lower 128 KiB (.lowtext):

      Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         00000000  00800200  0003f16e  0003f1e2  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  1 .lowtext      0001f10c  00000000  00000000  00000074  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .hightext     00020062  0001f10c  0001f10c  0001f180  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE

The objdump confirms that the rest of text (.hightext) follows without
one wasted byte. (There's one pad byte inside .hightext, if alignment is
needed, obviously.)

If we use some __flashN, perhaps because we have had to shovel a great
pile of .progmem.data* into .progmem2.data* and .progmem3.data, to pull
.lowtext back under 128 KiB, then the tools can now show us exactly what
is going on:

$ avr-gcc -T avr6.x-new -Wl,-Map,flash.map -o flash.elf -DSTUBS=10 \
  -DP0=0x1fe00 -DP2=0x10000 -DP3=0x87ff -DTEXT=0x20000 -mmcu=atmega2560 \
  flash.sx

   Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         00000000  00800200  00058862  00058802  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  1 .lowtext      0001ff0c  00000000  00000000  00000094  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .flash2       00010000  00020000  00020000  0001ffa0  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .flash3       000087ff  00030000  00030000  0002ffa0  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .hightext     00020063  000387ff  000387ff  0003879f  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE

We can see the margin at the 128 KiB limit:
(.flash2 VMA) - (.lowtext size) - 0
  00020000    -    0001ff0c          = 0xf4

We see that .flash2 is full, and .flash3 is about half full.
We see that no space is wasted between .flash3 and the rest of our code
in .hightext. Also, .data LMA immediately follows .hightext.

I.e. it is packed as tightly as the requirements allow.

If the boss asks "What's our memory situation? Can we add xxx features?",
this one command gives an instant summary of the strengths and
weaknesses of where we've stuffed things. If the full .lowtext is due to
.trampolines and other unavoidable stuff, then we can't just say yes to
lots more code.

Naturally, the "-j" arguments to avr-objcopy will differ a bit from
those used with an ATtiny15, but any serious software developer uses a
makefile target to produce the download file. After all, a manually
entered command of that sort is far too unreliable to be used for
software delivery.

Nevertheless, to handle a variety of .flashN output sections with a
shorter and invariant objcopy command line, it might be more attractive
to just use "-R .stab -R .stabstr", to let everything else through. That
may be usable across a variety of AVRs, without modification. (Not
tried, but "should just work".)

Anyway, I'm in the discussion until Wednesday, then incommunicado until
mid to late January. Perhaps the choice of future path will have been
mulled over by then.

My vote is to favour continued development of the tested successful
script candidate over the repeated failures of the other. Its current
effectiveness and resilience bodes well for unanticipated future
enhancements. Any tweaking requests can be knocked over then. (There
does not seem to be any hurry for any of this.)

Erik

¹ $ avr-gcc -T avr6.x-new -Wl,-Map,flash.map -o flash.elf -DSTUBS=10 \
    -DTEXT=0x10000 -mmcu=atmega2560 flash.sx

   Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         00000000  00800200  00010146  000101ba  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  1 .lowtext      000000e4  00000000  00000000  00000074  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .hightext     00010062  000000e4  000000e4  00000158  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE

Plainly, .lowtext, .hightext, and .data LMA all fit within 0x20000.
Add stuff, and it expands as elegantly as can be desired, I think.

-- 
(5)  It is always possible to agglutinate multiple separate problems
     into a single complex interdependent solution. In most cases
     this is a bad idea.                                    RFC-1925