Fix transaction aborting

During compilation, a java.lang.InternalError is used to indicate
that class initialization failed and the enclosing transaction
should be aborted and the changes rolled back. However there is
nothing preventing the code executed from a class initializer from
catching that exception (like catching Throwable and ignore it).
Therefore we may return from the class initializer with no pending
exception, even if the transaction was aborted, and not rollback
the changes properly.

To fix this, we now rely on the new Transaction::aborted_ field to
know whether a transaction aborted. When returning from the class
initializer without pending exception, we now check wether we aborted
the enclosing transaction. If that's the case, we set the status of
the class to kStatusError and throw a new java.lang.InternalError
with the original abort message.

This CL also contains some cleanup:
- Renames Transaction::Abort to Transaction::Rollback which is less
ambiguous and more reflect what is done.
- Moves the code throwing the java.lang.InternalError exception into
the Transaction::ThrowInternalError method so we do not duplicate
code. Now we may abort transaction more than once (because we may
have caught the java.lang.InternalError then execute code causing
new transaction abort), we only keep the first abort message to
throw the exception.
- Updates transaction_test with more cases and more checks.
- Bumps oat version to force recompilation with this fix.

Bug: 19202032
Change-Id: Iedc6969528a68bbdf3123146e990df4dbc57834b

1 files changed, 7 insertions, 0 deletions

diff --git a/runtime/runtime.h b/runtime/runtime.h
index c5a8739904..118c838397 100644
--- a/runtime/runtime.h
+++ b/runtime/runtime.h
@@ -455,6 +455,13 @@ class Runtime {
   }
   void EnterTransactionMode(Transaction* transaction);
   void ExitTransactionMode();
+  bool IsTransactionAborted() const;
+
+  void AbortTransactionAndThrowInternalError(Thread* self, const std::string& abort_message)
+      SHARED_LOCKS_REQUIRED(Locks::mutator_lock_);
+  void ThrowInternalErrorForAbortedTransaction(Thread* self)
+      SHARED_LOCKS_REQUIRED(Locks::mutator_lock_);
+
   void RecordWriteFieldBoolean(mirror::Object* obj, MemberOffset field_offset, uint8_t value,
                                bool is_volatile) const;
   void RecordWriteFieldByte(mirror::Object* obj, MemberOffset field_offset, int8_t value,