vector with large number of tiny objects causes problems

Hello,

I'm writing an application that operates on images using Visual C++ 7.1.
Since it should be able to handle many different kinds of pixel formats,
I wrote a color class template like

template<typename T>
class Color {
T rgb[3];
};

The images are vectors of colors like

std::vector< Color<unsigned char> > image;

This seems to work fine for small images, but larger ones (ie. 10 images
at 1024x1024) tend to result in application crashes when an image is
destroyed. An "unhandled exception: user breakpoint" appears in
ntdll.dll, the last known function on the call stack is
std::_Destroy_range() (defined in xmemory). Another common error are bad
alloc exceptions (caused by the operator new on vector.push_back()
calls), even though only about 100mb (on a 1gb machine) are allocated.

Does VC7.1 have a problem with large numbers of tiny objects? The Color
class does not introduce any size overhead compared to a simple array of
built-in types, so I wonder wether there are any other limitations.


Malte

Alexander
Fri Oct 01 21:07:30 CDT 2004

You may be writing beyond the allocated size. If you check the debugger
window, you'll see a message from the heap manager.

"Malte Clasen" <news@copro.org> wrote in message
news:2s69rbF1h3luqU1@uni-berlin.de...
> Hello,
>
> I'm writing an application that operates on images using Visual C++ 7.1.
> Since it should be able to handle many different kinds of pixel formats, I
> wrote a color class template like
>
> template<typename T>
> class Color {
> T rgb[3];
> };
>
> The images are vectors of colors like
>
> std::vector< Color<unsigned char> > image;
>
> This seems to work fine for small images, but larger ones (ie. 10 images
> at 1024x1024) tend to result in application crashes when an image is
> destroyed. An "unhandled exception: user breakpoint" appears in ntdll.dll,
> the last known function on the call stack is std::_Destroy_range()
> (defined in xmemory). Another common error are bad alloc exceptions
> (caused by the operator new on vector.push_back() calls), even though only
> about 100mb (on a 1gb machine) are allocated.
>
> Does VC7.1 have a problem with large numbers of tiny objects? The Color
> class does not introduce any size overhead compared to a simple array of
> built-in types, so I wonder wether there are any other limitations.
>
>
> Malte

Tim
Sat Oct 02 17:29:09 CDT 2004

Malte Clasen <news@copro.org> wrote:
>
>I'm writing an application that operates on images using Visual C++ 7.1.
>Since it should be able to handle many different kinds of pixel formats,
>I wrote a color class template like
>
>template<typename T>
>class Color {
> T rgb[3];
>};
>
>The images are vectors of colors like
>
>std::vector< Color<unsigned char> > image;
>
>This seems to work fine for small images, but larger ones (ie. 10 images
>at 1024x1024) tend to result in application crashes when an image is
>destroyed. An "unhandled exception: user breakpoint" appears in
>ntdll.dll, the last known function on the call stack is
>std::_Destroy_range() (defined in xmemory). Another common error are bad
>alloc exceptions (caused by the operator new on vector.push_back()
>calls), even though only about 100mb (on a 1gb machine) are allocated.

Are you pre-allocating the size you need using vector::reserve? If not,
you will run out of memory long before the address space is used up because
of heap fragmentation.

If you build up a vector element by element using push_back, the vector
doubles in size each time it fills up. That leaves behind unused blocks
that cannot be reused by that vector.

>Does VC7.1 have a problem with large numbers of tiny objects? The Color
>class does not introduce any size overhead compared to a simple array of
>built-in types, so I wonder wether there are any other limitations.

I don't think the problem is Color. I think the problem is vector.
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc

Malte
Sun Oct 03 11:39:39 CDT 2004

Tim Roberts wrote:
> Are you pre-allocating the size you need using vector::reserve? If not,
> you will run out of memory long before the address space is used up because
> of heap fragmentation.

Alexander Grigoriev wrote:
> You may be writing beyond the allocated size. If you check the
> debugger window, you'll see a message from the heap manager.

Thanks for both answers. On the one hand, I actually wrote beyond the
end of the image which caused the problems while deallocating, on the
other hand I had one method that filled the image using push_back. Now
both problems are solved and the program is running fine :)


Malte