ThingWorx Analytics server throws error "java.lang.IllegalStateException: Reading from cache:nio:<microservice>db.mv.db failed;"
Applies To
- ThingWorx Analytics 8.1 to 9.5
Description
- Analytics Builder can not create a model
- Analytics Builder model Submit button is grayed out
- Analytics entities (including Gateway thing) are not created in the ThingWorx Server
- Verifying configuration to Analytics Server may throw below message/output

- Big number of log files get created everyday
- ThingWorx Composer may show the following error

- CentOS VM /opt/ThingWorxAnalyticsServer/data/log throws the following error
Reading from cache:nio:/opt/ThingWorxAnalyticsServer/data/db/data-service.mv.db failed
ERROR [2017-12-01 02:28:40,057] org.apache.tomcat.jdbc.pool.ConnectionPool: Unable to create initial connections of pool.
java.io.EOFException: null
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:431)
... 48 common frames omitted
Causing: java.lang.IllegalStateException: Reading from cache:nio:/opt/ThingWorxAnalyticsServer/data/db/data-service-db.mv.db failed; file length 499712 read length 1024 at 509093 [1.4.192/1]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:773)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:443)
at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
at org.h2.mvstore.Page.read(Page.java:190)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1944)
at org.h2.mvstore.MVMap.readPage(MVMap.java:736)
at org.h2.mvstore.Page.getChildPage(Page.java:217)
at org.h2.mvstore.Cursor.fetchNext(Cursor.java:150)
at org.h2.mvstore.Cursor.next(Cursor.java:50)
at org.h2.mvstore.MVStore.loadChunkMeta(MVStore.java:692)
at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:672)
at org.h2.mvstore.MVStore.<init>(MVStore.java:355)
at org.h2.mvstore.MVStore$Builder.open(MVStore.java:2893)
at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:154)
... 36 common frames omitted
Causing: org.h2.jdbc.JdbcSQLException: IO Exception: null [90028-192]
- Other type of errors / behaviours observed:
- Unable to run Training Model
- Models training terminate with Failed status
- Unable to run Signals or Profiles
- Models > Job Details shows java.net.ConnectException: Connection refused: connect Connection refused
- ScriptErrorLog.log reports
- Execution error in service script [GetModelListStatisticsAMS] :: [ValidationMicroserver] could not retrieve result for jobId [08c146dc-587d-4bb8-82e1-8a4538a82692]: Unable to reach server at "results: […]
- result.log files shows
! ... 54 common frames omitted
! Causing: java.lang.IllegalStateException: Reading from cache:nio:C:/Program Files (x86)/ThingWorxAnalytics/Thingworx_Analytics_Server/data/db/results.mv.db failed; file length 425984 read length 1024 at 427324 [1.4.192/1]
! at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:773)
! at org.h2.mvstore.DataUtils.readFully(DataUtils.java:443)
! at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
! at org.h2.mvstore.Page.read(Page.java:190)
! at org.h2.mvstore.MVStore.readPage(MVStore.java:1944)
! at org.h2.mvstore.MVMap.readPage(MVMap.java:736)
! at org.h2.mvstore.Page.getChildPage(Page.java:217)
! at org.h2.mvstore.MVMap.binarySearch(MVMap.java:468)
- Anomaly detection / alert ends in FAILED state
- ApplicationLog.log file reports
[L: ERROR] [O: c.t.a.t.ThingWatcherInternal] [I: ] [U: YYYY] [S: ] [P: ] [T: TWEventProcessor-12] Returning a FAILED state for TimedValue [timestamp=1593160124755, value=7.0]. ThingWatcherMessage [timestamp=2020-06-26T10:28:45.869, severity=ERROR, state=ThingWatcherInternalState [internal=TRAINING, external=TRAINING], messageCode=WAT9999E, messageText=Unexpected exception. {Throwable=[GenericHTTPException: java.net.ConnectException: Connection refused: connect}
[L: ERROR] [O: c.t.s.s.a.a.AnomalyMonitor] [I: ] [U: YYYY] [S: ] [P: ] [T: TWEventProcessor-12] Anomaly Alert [XXXX] Failed when in [TRAINING] State. Thingwatcher Error Code :: [WAT9999E] Message :: Unexpected exception. {Throwable=[GenericHTTPException: java.net.ConnectException: Connection refused: connect}]
[L: ERROR] [O: c.t.s.s.a.a.AnomalyMonitor] [I: ] [U: YYYY] [S: ] [P: ] [T: TWEventProcessor-6] Exception occurred sending data to ThingWatcherId : [XXXX]. Exception :: [null]
- <ThingWorxAnalyticsServer>/data/logs/async.log reports
- Note: depending on the database file impacted (above examples are for the data and results microservice datafile), the behaviour and errors can vary