x

Best practices for using Fuzzy-grouping & Fuzzy-Look up

Where can I find best practices for using Fuzzy grouping & Fuzzy look up components? I need to process 10 000 000 records but I can't, I get an out-of-memory message. I have 8 MB in RAM.

Any idea or suggestion?

Thanks.

more ▼

asked Oct 23 '09 at 01:13 PM in Default

felix gravatar image

felix
11 1 1 1

This is slightly offtopic, but if you are handling very large datasets with any kind of frequency or concern for performance, you may want to consider more than 8GB of RAM. With databases in general and large datasets in particular, increasing the RAM is one of the best overall ways to improve performance. (Of course this should always be in addition to, not in place of, optimizing your code).
Jul 15 '10 at 04:19 PM TimothyAWiseman
(comments are locked)
10|1200 characters needed characters left

1 answer: sort voted first

Hi Felix,

you can take a look into this article. it will explain you how to change the buffer manager settings. this amount of rows are to big to process, so you're getting without memory/buffer available to finish the processing.. you can try to increase the DefaultBufferSize property of your package and see if this solves your problem

more ▼

answered Oct 23 '09 at 03:20 PM

Felipe Ferreira gravatar image

Felipe Ferreira
306 2

(comments are locked)
10|1200 characters needed characters left
Your answer
toggle preview:

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

New code box

There's a new way to format code on the site - the red speech bubble logo will automatically format T-SQL for you. The original code box is still there for XML, etc. More details here.

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

SQL Server Central

Need long-form SQL discussion? SQLserverCentral.com is the place.

Topics:

x898
x3

asked: Oct 23 '09 at 01:13 PM

Seen: 2637 times

Last Updated: Oct 23 '09 at 06:01 PM