Back to Articles
The following assumptions are used in these deployment scenarios:
  • Multidatacenter functionality is enabled on both Site A and Site B clusters at the time of installation.

  • For a new installation, Site B must be installed first.

  • Network connectivity exists between Cassandra in Site A and Site B.
  • Cassandra is configured as a single cluster with two datacenters. Cassandra nodes in Site A are configured as one datacenter, and Cassandra nodes in Site B are a separate datacenter.
  • Each site has at least 3 Cassandra nodes.

  • Cassandra nodes in both datacenters are started and running as part of normal operation (non-failover mode). Data is being replicated between Cassandra in Site A and Site B.
  • Network connectivity exists between Insight Views in Site A and Site B.

  • Singlestore nodes in both datacenters are started and running as part of normal operation (non-failover mode). Data is being replicated between Singlestore in Site A and Site B.

  • DOCSIS devices in Region 1 and Region 2 are accessible from either data center.
  • During normal operation (non-failover mode), ServAssure Pollers in Site A are polling DOCSIS network elements in Site A region and ServAssure Pollers in Site B are polling DOCSIS network elements in Site B region.
  • ServAssure Pollers located in Site B can poll DOCSIS network elements in Site A in a Site A failure scenario.
  • Hosts used for running Ansible playbooks are able to configure hosts in both Site A and Site B (can connect to ServAssure NXT, ServAssure Collection Platform, and ServAssure NXT Insight Views hosts in both environments via SSH).
  • Separate Ansible inventory files are maintained for configuring Site A and Site B. Ansible inventory files are updated with all requirements for the current release.
  • The Collection Manager registration.db is copied on to Collection Managers at both Site A and Site B as a database "backup".
  • A load balancer is installed and used in front of the ServAssure NXT UI to simplify topology import and to direct Alarm Dashboard and Tableau user access.
  • Conversion of existing ServAssure NXT Insight Views data covering several weeks' collection will require significant time to replicate to the other site. It is possible to replicate in the background without affecting the primary site.
If a Cassandra outage occurs and Cassandra in Site B site is down for a period of time, where down means Cassandra software has not been running or network connectivity between the two Cassandra datacenters has been interrupted, then the following applies:
  • If the Cassandra outage is shorter than 3 hours:

    Cassandra will store hints for up to 3 hours for a node that is down. It will replay the missed writes when the node rejoins the Cassandra cluster.

  • If the Cassandra outage is longer than 3 hours:
    Nodes that have been down for longer than 3 hours must be rebuilt. Run
    'nodetool rebuild -- AlarmCentral_Cluster'
    on these nodes. To rebuild:
    1. Ensure that Cassandra is shut down on the node being rebuilt.
    2. Change “auto_bootstrap: false” in /etc/cassandra/conf/cassandra.yml (may have to add the property if it does not exist).
    3. Remove Cassandra commit logs, save_cache directories, and SSTables.
    4. Start Cassandra on the node being rebuilt.
    5. Run "nodetool rebuild -- AlarmCentral_Cluster" on the node(s) being rebuilt. AlarmCentral_Cluster is the name of the source cluster you are getting the data from. It might be a different name.
    6. Change back “auto_bootstrap: true” in /etc/cassandra/conf/cassandra.yml.