tag:status.daysmartvet.com,2005:/historyDaySmartvet Status - Incident History2024-03-28T05:50:58-07:00DaySmartvettag:status.daysmartvet.com,2005:Incident/202458582024-03-15T07:05:53-07:002024-03-15T07:05:53-07:00IDEXX Outage<p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>07:05</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Mar <var data-var='date'>14</var>, <var data-var='time'>11:00</var> PDT</small><br><strong>Update</strong> - DEXX is currently investigating the issue</p><p><small>Mar <var data-var='date'>14</var>, <var data-var='time'>10:59</var> PDT</small><br><strong>Investigating</strong> - IDEXX is currently investigating the issue</p>tag:status.daysmartvet.com,2005:Incident/175838792023-07-05T09:35:46-07:002023-07-05T09:35:46-07:00Dymo Printer Issue<p><small>Jul <var data-var='date'> 5</var>, <var data-var='time'>09:35</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>15</var>, <var data-var='time'>14:33</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jun <var data-var='date'>15</var>, <var data-var='time'>08:17</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jun <var data-var='date'>15</var>, <var data-var='time'>08:17</var> PDT</small><br><strong>Investigating</strong> - We're experiencing an issue with labels printing upside down for DYMO printers. We’re working to identify the cause and resolve it. We'll share an update shortly. Thanks for your patience.</p>tag:status.daysmartvet.com,2005:Incident/101694612022-06-07T20:01:28-07:002022-06-07T20:01:28-07:00Antech print requisition forms not working - Antech is aware of the issue and is working on a fix<p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>20:01</var> PDT</small><br><strong>Resolved</strong> - The issue appears to have been resolved on the Antech side. Closing.</p><p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>10:15</var> PDT</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>09:50</var> PDT</small><br><strong>Identified</strong> - Antech is working on the fix for this issue.</p>tag:status.daysmartvet.com,2005:Incident/89761812022-01-01T00:00:30-08:002022-01-01T00:00:30-08:00Database maintenance<p><small>Jan <var data-var='date'> 1</var>, <var data-var='time'>00:00</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>23:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>18:52</var> PST</small><br><strong>Scheduled</strong> - We will be doing planned database maintenance during this time and some part of the application may be intermittently available.</p>tag:status.daysmartvet.com,2005:Incident/88558762021-12-15T08:27:28-08:002022-05-24T08:59:22-07:00Amazon Web Services Internet connectivity issues<p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>08:27</var> PST</small><br><strong>Resolved</strong> - From Amazon Web Services:<br /><br />8:14 AM PST We have resolved the issue affecting Internet connectivity to the US-WEST-2 Region. Connectivity within the region was not affected by this event. The issue has been resolved and the service is operating normally.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>08:10</var> PST</small><br><strong>Identified</strong> - From Amazon Web Services:<br /><br />7:43 AM PST We are investigating Internet connectivity issues to the US-WEST-2 Region.<br />8:01 AM PST We have identified the root cause of the Internet connectivity to the US-WEST-2 Region and have taken steps to restore connectivity. We have seen some improvement to Internet connectivity in the last few minutes but continue to work towards full recovery.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>08:04</var> PST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>08:03</var> PST</small><br><strong>Investigating</strong> - Our cloud provider has been having widespread connectivity issues starting at about 7:15AM PDT that resulted in many people being unable to log in. We are tracking the issue.</p>tag:status.daysmartvet.com,2005:Incident/81446132021-10-03T16:46:38-07:002022-05-24T08:59:22-07:00Records not creating when adding from a bundle<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>16:46</var> PDT</small><br><strong>Resolved</strong> - The root cause was a database index that was not being called appropriately, resulting in very long processing times. This has been patched and verified to be fixed. We're sorry for any inconvenience caused.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>14:54</var> PDT</small><br><strong>Investigating</strong> - We have received reports of issues where adding records from a bundle is not completing as expected and stopping the process mid-flow results in invoiced items with no records having been created. We have alerted our engineering team to the issue and are working to determine the root cause. We will update this incident shortly with more information.</p>tag:status.daysmartvet.com,2005:Incident/65455382021-03-16T11:48:17-07:002022-05-24T08:59:22-07:00Antech issues<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>11:48</var> PDT</small><br><strong>Resolved</strong> - Antech let us know that the issue has been resolved. We verified with a reporting clinic that the same is true. Resolving this issue.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>10:22</var> PDT</small><br><strong>Investigating</strong> - Antech has been experiencing issues this morning with their online integration platform. The integration has been down intermittently as a result. We've notified our Antech technical contact and are waiting for an update.</p>tag:status.daysmartvet.com,2005:Incident/64831162021-03-12T15:15:13-08:002022-05-24T08:59:22-07:00Antech integration is offline<p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>15:15</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 8</var>, <var data-var='time'>20:27</var> PST</small><br><strong>Update</strong> - We heard back from Antach and they're back online. Resolving this issue.</p><p><small>Mar <var data-var='date'> 8</var>, <var data-var='time'>13:35</var> PST</small><br><strong>Investigating</strong> - We have received multiple reports of the Antech integration not working. The issue presents as a Java error on lab record save. Additionally, there have been reports from clinics that they are unable to access the antechonline.com website either. We believe this to be an issue on the Antech side and we have escalated the issue to their technical team for review. Will update this ticket when we know more.</p>tag:status.daysmartvet.com,2005:Incident/62388472021-02-10T16:35:29-08:002022-05-24T08:59:22-07:00IDEXX Integration not working<p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>16:35</var> PST</small><br><strong>Resolved</strong> - IDEXX has informed us that the issue has been resolved.</p><p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>12:07</var> PST</small><br><strong>Update</strong> - IDEXX sent a message to their partners letting us know that they will be providing hourly updates related to this downtime. To be more clear about the issue: requisition forms cannot be printed due to a processing issue at IDEXX. We will provide updates as received from IDEXX.</p><p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>11:53</var> PST</small><br><strong>Investigating</strong> - IDEXX has informed us that they are experiencing a global issue with their integrated labs. We will provide an update as soon as we hear more from IDEXX.</p>tag:status.daysmartvet.com,2005:Incident/61114342021-01-26T10:48:41-08:002022-05-24T08:59:22-07:00IDEXX Labs integration returns error / blank screen<p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>10:48</var> PST</small><br><strong>Resolved</strong> - We have determined the cause of this issue and have reached out to affected clinics to resolve.</p><p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>11:02</var> PST</small><br><strong>Investigating</strong> - Several clinics have reported that when a lab is created in Vetter and the "next" button is clicked, the screen that normally shows an IDEXX page is now showing blank or an error message. Results will still come back to Unassigned Labs, however they will not return to the patient record. We have escalated this to IDEXX and we're waiting to hear back regarding this issue.</p>tag:status.daysmartvet.com,2005:Incident/60798212021-01-21T11:37:30-08:002022-05-24T08:59:22-07:00Zoetis Fuse integration not fully functional<p><small>Jan <var data-var='date'>21</var>, <var data-var='time'>11:37</var> PST</small><br><strong>Resolved</strong> - Zoetis has notified us that the issue has been resolved.</p><p><small>Jan <var data-var='date'>21</var>, <var data-var='time'>10:44</var> PST</small><br><strong>Investigating</strong> - The Vetter Software integration with Zoetis is not fully functional. Zoetis is aware of the issue and they're working to get their bi-directional connection with Vetter back online.</p>tag:status.daysmartvet.com,2005:Incident/52349902020-10-02T11:24:16-07:002022-05-24T08:59:22-07:00Slowness<p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:24</var> PDT</small><br><strong>Resolved</strong> - Issue has been resolved.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>09:11</var> PDT</small><br><strong>Monitoring</strong> - We have upgraded our database server and the contention issues appear to be resolved. We are monitoring to ensure that this remains true.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>08:52</var> PDT</small><br><strong>Update</strong> - We have identified the issue and we're working on a second fix now. Another update in 10min.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>08:05</var> PDT</small><br><strong>Identified</strong> - We are experiencing application slowness as a result of a database contention issue. We have identified the issue and we're working to get it resolved.</p>tag:status.daysmartvet.com,2005:Incident/50440782020-09-08T10:46:23-07:002022-05-24T08:59:22-07:00Database slowness<p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>10:46</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>09:42</var> PDT</small><br><strong>Identified</strong> - We have identified the root cause of the slowness and we're working to get it addressed. Will have another update shortly.</p><p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>09:29</var> PDT</small><br><strong>Investigating</strong> - We're experiencing an issue with database slowness. We’re working to identify the cause and resolve it. We'll send an update shortly. Thanks for your patience.</p>tag:status.daysmartvet.com,2005:Incident/48489032020-08-14T12:35:48-07:002022-05-24T08:59:22-07:00Antech processing issues<p><small>Aug <var data-var='date'>14</var>, <var data-var='time'>12:35</var> PDT</small><br><strong>Resolved</strong> - This issue appears to be resolved. Labs that weren't being loaded now appear and there have been no other reports of issues thus far today.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>16:01</var> PDT</small><br><strong>Investigating</strong> - We are still investigating the issue.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>14:03</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>13:51</var> PDT</small><br><strong>Investigating</strong> - We are currently experiencing an issue with the Antech integration where the connection on their end is timing out. We escalated to Antech and we are awaiting a response. We will update as soon as we have word from Antech.</p>tag:status.daysmartvet.com,2005:Incident/44412712020-06-23T08:58:17-07:002022-05-24T08:59:22-07:00Antech Integration Down<p><small>Jun <var data-var='date'>23</var>, <var data-var='time'>08:58</var> PDT</small><br><strong>Resolved</strong> - Antech servers are back online and the integration should work normally.</p><p><small>Jun <var data-var='date'>23</var>, <var data-var='time'>08:11</var> PDT</small><br><strong>Investigating</strong> - Antech servers are currently unavailable. The Antech team is checking into the issue and will let us know when resolved.</p>tag:status.daysmartvet.com,2005:Incident/42351452020-05-30T13:57:56-07:002022-05-24T08:59:22-07:00Errors connecting to Antech<p><small>May <var data-var='date'>30</var>, <var data-var='time'>13:57</var> PDT</small><br><strong>Resolved</strong> - This issue is now resolved</p><p><small>May <var data-var='date'>30</var>, <var data-var='time'>13:13</var> PDT</small><br><strong>Update</strong> - The issue appears to be on the Antech side and we're waiting for resolution. Will update again when we get word back from Antech.</p><p><small>May <var data-var='date'>30</var>, <var data-var='time'>09:03</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating and escalating this issue with Antech.</p>tag:status.daysmartvet.com,2005:Incident/35946902020-02-21T10:36:15-08:002022-05-24T08:59:22-07:00Patient history not loading<p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>10:36</var> PST</small><br><strong>Resolved</strong> - Fix has been implemented. Issue resolved.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>10:31</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>09:40</var> PST</small><br><strong>Investigating</strong> - We are currently investigating reports of patient histories not loading</p>tag:status.daysmartvet.com,2005:Incident/35944802020-02-21T09:34:31-08:002022-05-24T08:59:23-07:00Application slowness<p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>09:34</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>09:13</var> PST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>08:40</var> PST</small><br><strong>Investigating</strong> - We are currently investigating some reports of slowness. We will update shortly.</p>tag:status.daysmartvet.com,2005:Incident/33961692020-01-06T14:37:55-08:002022-05-24T08:59:23-07:00Quickbooks is offline<p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>14:37</var> PST</small><br><strong>Resolved</strong> - Quickbooks is back online.</p><p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>10:17</var> PST</small><br><strong>Identified</strong> - Quickbooks Online is down. Here's the link to the Quickbooks status page: http://status.quickbooks.intuit.com/<br /><br />Please disregard the re-enable Quickbooks integration emails until the Quickbooks Online product is active.</p>tag:status.daysmartvet.com,2005:Incident/33828212020-01-02T12:37:32-08:002022-05-24T08:59:23-07:00Application slowness<p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>12:37</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>12:37</var> PST</small><br><strong>Update</strong> - No further issues seen; marking resolved.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>10:32</var> PST</small><br><strong>Monitoring</strong> - We are still seeing a slightly increased load on our database but performance appears to have gone back to normal. We will monitor this issue for the next couple of hours.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>09:29</var> PST</small><br><strong>Investigating</strong> - We're checking into the cause of the slowness issue now and will have an update shortly.</p>tag:status.daysmartvet.com,2005:Incident/33533222019-12-28T11:30:19-08:002022-05-24T08:59:23-07:00Google Cloud Print issue<p><small>Dec <var data-var='date'>28</var>, <var data-var='time'>11:30</var> PST</small><br><strong>Resolved</strong> - This incident was resolved on Friday morning PST. Marking resolved.</p><p><small>Dec <var data-var='date'>24</var>, <var data-var='time'>16:23</var> PST</small><br><strong>Update</strong> - We heard back from Google and they have told us what we need to do to get the integration working again. Unfortunately the changes they made on their end were not communicated, and much larger in scope than we were led to believe. We should have another update on Sunday, if not sooner.</p><p><small>Dec <var data-var='date'>23</var>, <var data-var='time'>22:12</var> PST</small><br><strong>Update</strong> - We are waiting for an update from Google regarding the verification of our new credentials. In the meantime, we've enabled the ability to print from PDF. We have included instructions for setting this up in an email to clinic administrators. Will update this issue in the morning when we (hopefully) hear back from Google.</p><p><small>Dec <var data-var='date'>23</var>, <var data-var='time'>06:39</var> PST</small><br><strong>Investigating</strong> - Google Cloud Print can't be verified the normal way. We're working with Google to get this resolved and will provide an update shortly.</p>tag:status.daysmartvet.com,2005:Incident/33281442019-12-16T10:46:04-08:002022-05-24T08:59:23-07:00Application slowness due to partner integration<p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>10:46</var> PST</small><br><strong>Resolved</strong> - Marking as resolved.</p><p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>10:05</var> PST</small><br><strong>Update</strong> - We pushed a code update that resolved this issue. We are monitoring post-update.</p><p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>09:46</var> PST</small><br><strong>Investigating</strong> - We are seeing an issue related to a partner integration that's causing performance issues in Vetter. Our engineering team is checking into the dependency now and we'll have an update shortly.</p>tag:status.daysmartvet.com,2005:Incident/33063802019-12-10T12:04:22-08:002022-05-24T08:59:23-07:00Abaxis service outage<p><small>Dec <var data-var='date'>10</var>, <var data-var='time'>12:04</var> PST</small><br><strong>Resolved</strong> - Abaxis issues are now resolved.</p><p><small>Dec <var data-var='date'>10</var>, <var data-var='time'>07:38</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Dec <var data-var='date'>10</var>, <var data-var='time'>07:38</var> PST</small><br><strong>Investigating</strong> - Abaxis is reporting an outage affecting results transferring to Vetter Software that began around 5:54AM PST.</p>tag:status.daysmartvet.com,2005:Incident/32777232019-12-02T13:35:46-08:002022-05-24T08:59:23-07:00Degraded application performance<p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>13:35</var> PST</small><br><strong>Resolved</strong> - Servers have continued to perform normally post-update. Marking resolved.</p><p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>13:16</var> PST</small><br><strong>Monitoring</strong> - Server performance is recovering and application performance has improved. We will continue to monitor performance to ensure the fix holds.</p><p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>12:59</var> PST</small><br><strong>Investigating</strong> - We have an issue related to application server CPU usage. We are in the process of standing up more application servers to better distribute load while we dig into the root cause. Another update will be provided in 10min.</p>tag:status.daysmartvet.com,2005:Incident/32301262019-11-18T20:08:47-08:002022-05-24T08:59:23-07:00Application slowness<p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>20:08</var> PST</small><br><strong>Resolved</strong> - This issue was resolved earlier this afternoon.</p><p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>16:09</var> PST</small><br><strong>Investigating</strong> - We appear to be experiencing an issue similar to what we saw earlier today, where performance has slowed as a result of increased database activity. We're checking into the cause of the issue and will provide an update shortly.</p>