Article - CS274246

ThingWorx Analytics server throws error "java.lang.IllegalStateException: Reading from cache:nio:<microservice>db.mv.db failed;"

Modified: 14-May-2024   


Applies To

  • ThingWorx Analytics 8.1 to 9.5

Description

  • Analytics Builder can not create a model
  • Analytics Builder model Submit button is grayed out 
  • Analytics entities (including Gateway thing) are not created in the ThingWorx Server
  • Verifying configuration to Analytics Server may throw below message/output
Selected Analytics Gateway is not connected
Analytics server is not connected
 
  • Big number of log files get created everyday
  • ThingWorx Composer may show the following error
Execution error in service script [TW_ML_Helper GetDatasetsAMS]: Unable to Invoke Service GetDatasetsAMS on TW_ML_Helper:Got error code [502]
Composer
 
  • CentOS VM /opt/ThingWorxAnalyticsServer/data/log throws the following error

Reading from cache:nio:/opt/ThingWorxAnalyticsServer/data/db/data-service.mv.db failed
ERROR [2017-12-01 02:28:40,057] org.apache.tomcat.jdbc.pool.ConnectionPool: Unable to create initial connections of pool.
 java.io.EOFException: null
 at org.h2.mvstore.DataUtils.readFully(DataUtils.java:431)
 ... 48 common frames omitted
 Causing: java.lang.IllegalStateException: Reading from cache:nio:/opt/ThingWorxAnalyticsServer/data/db/data-service-db.mv.db failed; file length 499712 read length 1024 at 509093 [1.4.192/1]
 at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:773)
 at org.h2.mvstore.DataUtils.readFully(DataUtils.java:443)
 at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
 at org.h2.mvstore.Page.read(Page.java:190)
 at org.h2.mvstore.MVStore.readPage(MVStore.java:1944)
 at org.h2.mvstore.MVMap.readPage(MVMap.java:736)
 at org.h2.mvstore.Page.getChildPage(Page.java:217)
 at org.h2.mvstore.Cursor.fetchNext(Cursor.java:150)
 at org.h2.mvstore.Cursor.next(Cursor.java:50)
 at org.h2.mvstore.MVStore.loadChunkMeta(MVStore.java:692)
 at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:672)
 at org.h2.mvstore.MVStore.<init>(MVStore.java:355)
 at org.h2.mvstore.MVStore$Builder.open(MVStore.java:2893)
 at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:154)
 ... 36 common frames omitted
 Causing: org.h2.jdbc.JdbcSQLException: IO Exception: null [90028-192]

  • Other type of errors / behaviours observed:
    • Unable to run Training Model
    • Models training terminate with Failed status
    • Unable to run Signals or Profiles
    • Models > Job Details shows java.net.ConnectException: Connection refused: connect Connection refused
    • ScriptErrorLog.log reports
      • Execution error in service script [GetModelListStatisticsAMS] :: [ValidationMicroserver] could not retrieve result for jobId [08c146dc-587d-4bb8-82e1-8a4538a82692]: Unable to reach server at "results: […]
    • result.log files shows
! java.io.EOFException: null
! at org.h2.mvstore.DataUtils.readFully(DataUtils.java:431)
! ... 54 common frames omitted
! Causing: java.lang.IllegalStateException: Reading from cache:nio:C:/Program Files (x86)/ThingWorxAnalytics/Thingworx_Analytics_Server/data/db/results.mv.db failed; file length 425984 read length 1024 at 427324 [1.4.192/1]
! at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:773)
! at org.h2.mvstore.DataUtils.readFully(DataUtils.java:443)
! at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
! at org.h2.mvstore.Page.read(Page.java:190)
! at org.h2.mvstore.MVStore.readPage(MVStore.java:1944)
! at org.h2.mvstore.MVMap.readPage(MVMap.java:736)
! at org.h2.mvstore.Page.getChildPage(Page.java:217)
! at org.h2.mvstore.MVMap.binarySearch(MVMap.java:468)
 
  • Anomaly detection / alert ends in FAILED state
    • ApplicationLog.log file reports
Error submitting the training request: java.net.ConnectException: Connection refused: connect
 [L: ERROR] [O: c.t.a.t.ThingWatcherInternal] [I: ] [U: YYYY] [S: ] [P: ] [T: TWEventProcessor-12] Returning a FAILED state for TimedValue [timestamp=1593160124755, value=7.0]. ThingWatcherMessage [timestamp=2020-06-26T10:28:45.869, severity=ERROR, state=ThingWatcherInternalState [internal=TRAINING, external=TRAINING], messageCode=WAT9999E, messageText=Unexpected exception. {Throwable=[GenericHTTPException: java.net.ConnectException: Connection refused: connect}
 [L: ERROR] [O: c.t.s.s.a.a.AnomalyMonitor] [I: ] [U: YYYY] [S: ] [P: ] [T: TWEventProcessor-12] Anomaly Alert [XXXX] Failed when in [TRAINING] State. Thingwatcher Error Code :: [WAT9999E]  Message :: Unexpected exception. {Throwable=[GenericHTTPException: java.net.ConnectException: Connection refused: connect}]
 [L: ERROR] [O: c.t.s.s.a.a.AnomalyMonitor] [I: ] [U: YYYY] [S: ] [P: ] [T: TWEventProcessor-6] Exception occurred sending data to ThingWatcherId : [XXXX]. Exception :: [null]

 
  • <ThingWorxAnalyticsServer>/data/logs/async.log reports
 java.lang.IllegalStateException: Reading from cache:nio:Thingworx_Analytics_Server/data/db/async-db.mv.db failed; file length 454656 read length 384 at 465946 [1.4.192/1]
  • Note: depending on the database file impacted (above examples are for the data and results microservice datafile), the behaviour and errors can vary
This is a printer-friendly version of Article 274246 and may be out of date. For the latest version click CS274246