We've run into a very strange issue when running against Oracle using System.Data.OracleClient.
As part of an import process that we're running we're doing a number of
validations on data that has already been inserted into our database. For
each validation within our import process we're logging errors by doing something
along the lines of:

INSERT INTO ImportMessages (ErrorContext, ErrorType, Timestamp, RowId)
SELECT Col1, 1, CurrentDate, RowIdColumn
FROM TableWithImportedRecords
WHERE {validation condition}

We subsequently update each invalid record in our bulk import table with
a simple UPDATE statement.

UPDATE BulkImport SET IsValid = 0 WHERE {validation condition}

We have approximately 15 validations that are run against the database in
sequence.

foreach(ImportValidation validation in validations) {
validation.Validate(); // this executes the two SQL statements above
errorCount = validation.ErrorCount;
warningCount = validation.WarningCount;
}

When we run our imports we're getting extemely inconsistent (random) behavior
from Oracle. When we execute the process against Sql Server everything works
reliably and consistently. As part of our debugging efforts we tracked things
down to the foreach() loop shown above and found that there is some sort
of timing issue when we execute our commands against Oracle. When we run
in the debugger with a breakpoint set on the errorCount line everything works
perfectly, but, as soon as we take the breakpoint off the code results in
extemely inconsistent and incorrect results. As an experiment we placed
a Thread.Sleep(500) statement inside the loop which resulted in everything
working properly.

What could be happening when those statements are executed against Oracle
that would cause them to return unexpected results when run in a tight foreach
loop? Are there any Oracle specific settings that I should look into (auto-commit?)?


Any ideas? I'm fresh out

Re: Oracle behaving strangely with multiple commands executing in a tight loop by Shawn

Shawn
Sun Jul 16 18:00:32 CDT 2006

Hello Steven,

Can you replicate the behavior with Oracle's Managed Provider (e.g. Oracle.Data.OracleClient)?

Thanks,
Shawn Wildermuth
Speaker, Author and C# MVP

> We've run into a very strange issue when running against Oracle using
> System.Data.OracleClient.
> As part of an import process that we're running we're doing a number
> of
> validations on data that has already been inserted into our database.
> For
> each validation within our import process we're logging errors by
> doing something along the lines of:
>
> INSERT INTO ImportMessages (ErrorContext, ErrorType, Timestamp, RowId)
> SELECT Col1, 1, CurrentDate, RowIdColumn
> FROM TableWithImportedRecords
> WHERE {validation condition}
> We subsequently update each invalid record in our bulk import table
> with a simple UPDATE statement.
>
> UPDATE BulkImport SET IsValid = 0 WHERE {validation condition}
>
> We have approximately 15 validations that are run against the database
> in sequence.
>
> foreach(ImportValidation validation in validations) {
> validation.Validate(); // this executes the two SQL statements above
> errorCount = validation.ErrorCount;
> warningCount = validation.WarningCount;
> }
> When we run our imports we're getting extemely inconsistent (random)
> behavior from Oracle. When we execute the process against Sql Server
> everything works reliably and consistently. As part of our debugging
> efforts we tracked things down to the foreach() loop shown above and
> found that there is some sort of timing issue when we execute our
> commands against Oracle. When we run in the debugger with a
> breakpoint set on the errorCount line everything works perfectly, but,
> as soon as we take the breakpoint off the code results in extemely
> inconsistent and incorrect results. As an experiment we placed a
> Thread.Sleep(500) statement inside the loop which resulted in
> everything working properly.
>
> What could be happening when those statements are executed against
> Oracle that would cause them to return unexpected results when run in
> a tight foreach loop? Are there any Oracle specific settings that I
> should look into (auto-commit?)?
>
> Any ideas? I'm fresh out.
>



Re: Oracle behaving strangely with multiple commands executing in a tight loop by Frans

Frans
Mon Jul 17 02:46:37 CDT 2006

Steven Eichert wrote:

> We've run into a very strange issue when running against Oracle using
> System.Data.OracleClient. As part of an import process that we're
> running we're doing a number of validations on data that has already
> been inserted into our database. For each validation within our
> import process we're logging errors by doing something along the
> lines of:
>
> INSERT INTO ImportMessages (ErrorContext, ErrorType, Timestamp, RowId)
> SELECT Col1, 1, CurrentDate, RowIdColumn
> FROM TableWithImportedRecords
> WHERE {validation condition}
>
> We subsequently update each invalid record in our bulk import table
> with a simple UPDATE statement.
>
> UPDATE BulkImport SET IsValid = 0 WHERE {validation condition}
>
> We have approximately 15 validations that are run against the
> database in sequence.
>
> foreach(ImportValidation validation in validations) {
> validation.Validate(); // this executes the two SQL statements above
> errorCount = validation.ErrorCount;
> warningCount = validation.WarningCount;
> }
>
> When we run our imports we're getting extemely inconsistent (random)
> behavior from Oracle. When we execute the process against Sql Server
> everything works reliably and consistently. As part of our debugging
> efforts we tracked things down to the foreach() loop shown above and
> found that there is some sort of timing issue when we execute our
> commands against Oracle. When we run in the debugger with a
> breakpoint set on the errorCount line everything works perfectly,
> but, as soon as we take the breakpoint off the code results in
> extemely inconsistent and incorrect results. As an experiment we
> placed a Thread.Sleep(500) statement inside the loop which resulted
> in everything working properly.
>
> What could be happening when those statements are executed against
> Oracle that would cause them to return unexpected results when run in
> a tight foreach loop? Are there any Oracle specific settings that I
> should look into (auto-commit?)? Any ideas? I'm fresh out.

Oracle uses MVCC, which is a concurrency scheme within a transaction:
if a transaction with DML statements is executed, the transaction can
manipulate data without affecting other threads. This means that
another command issued could be executed in parallel with the insert
and it won't see the inserted data until that transaction is completed.

On SqlServer, the update statement will block till the insert has been
completed.

Do you execute the insert and update in a single transaction? If not,
could you try to run it in a single transaction?

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------