Enhance energy utility operational effectivity utilizing good sensor information and Amazon QuickSight

This weblog submit is co-written with Steve Alexander at PG&E.

In at present’s quickly altering power panorama, energy disturbances trigger companies hundreds of thousands of {dollars} on account of service interruptions and energy high quality points. Massive utility territories make it troublesome to detect and find faults when energy outages happen, resulting in longer restoration occasions, recurring outages, and sad clients. Though it’s complicated and costly to modernize distribution networks, many utilities select to make use of their capital by the appliance of good sensor applied sciences. These good sensors are put in in chosen places on distribution networks to watch varied disturbances, equivalent to momentary and everlasting outages, line disturbances, voltage sags and surges. The sensors present analysts with fault waveforms and alerts along with graphical illustration of normal hundreds. Totally different communication infrastructure sorts equivalent to mesh community and mobile can be utilized to ship load data on a pre-defined schedule or occasion information in actual time to the backend servers residing within the utility UDN (Utility Information Community).

On this sequence of posts, we stroll you thru how we use Amazon QuickSight, a serverless, absolutely managed, enterprise intelligence (BI) service that allows data-driven resolution making at scale. QuickSight meets various analytics wants with trendy interactive dashboards, paginated studies, pure language queries, ML-insights, and embedded analytics, from one unified service.

On this first submit of the sequence, we present you the way information collected from good sensors is used for constructing automated dashboards utilizing QuickSight to assist distribution community engineers handle, keep and troubleshoot good sensors and carry out superior analytics to assist enterprise resolution making.

Present challenges in energy utility operations

To have a complete monitoring protection of the distribution networks, utilities usually deploy lots of, if not 1000’s, of good sensors. Much like every other gear or system, good sensors might encounter totally different points, equivalent to having faulty elements, carrying out over time, turning into out of date on account of technological advances, or struggling lack of communication on account of energy outages or low mobile sign protection. Managing such numerous gadgets may be difficult.

Moreover, based mostly on the use case, utilities usually apply sensor applied sciences from totally different distributors. Options from totally different distributors can range, equivalent to information protocols, codecs, native connectors, and communication media, which additional will increase the complexity of managing these good sensors.

To successfully remedy good sensor administration points and enhance operational effectivity, distribution engineers want a BI utility that’s easy to make use of and has a strong information processing and analytics engine. QuickSight supplies an excellent resolution to satisfy these enterprise wants.

Answer overview

The next extremely simplified architectural diagram illustrates the good sensor information assortment and processing. Good sensors ship information by way of mobile communication based mostly on a predefined schedule or triggered by real-time occasions. Information assortment and processing are dealt with by a third-party good sensor producer utility residing in Amazon Digital Non-public Cloud (Amazon VPC) non-public subnets behind a Community Load Balancer. Amazon Kinesis Information Streams interacts with the third-party utility by a local connection and conducts obligatory information transformation in actual time, and Amazon Kinesis Information Firehose shops the information in Amazon Easy Storage Service (Amazon S3) buckets. The AWS Glue Information Catalog comprises the desk definitions for the good sensor information sources saved within the S3 buckets. Amazon Athena runs queries utilizing a wide range of SQL statements on information saved in Amazon S3, and QuickSight is used for enterprise intelligence and information visualization.

After the good sensor’s information is collected and saved in Amazon S3 and is accessible by way of Athena, we are able to deal with constructing the next QuickSight dashboards for distribution community engineers:

  • Sensor standing dashboard – Analyze and monitor the standing of good sensors
  • Distribution community occasions dashboard – Analyze the operational data of the distribution networks


This resolution requires an energetic AWS account with the permission to create and modify AWS Id and Entry Administration (IAM) roles together with the next providers enabled:

  • Athena
  • AWS Glue
  • Kinesis Information Firehose
  • Kinesis Information Streams
  • Community Load Balancer
  • QuickSight
  • Amazon S3
  • Amazon VPC

Moreover, information assortment and information processing are purposeful blocks of the third-party good sensor producer utility. The good sensor utility resolution have to be already deployed in the identical AWS account and Area that you’ll use for the dashboards.

This resolution makes use of QuickSight SPICE (Tremendous-fast, Parallel, In-memory Calculation Engine) storage to enhance dashboard efficiency.

Sensor standing dashboard

When lots of or 1000’s of line sensors are put in, it’s vital for distribution engineers to know the standing of all good sensors regularly and repair points to make sure good sensors present real-time data for operator decision-making. Assuming a utility has 5,000 good sensors put in, even when only one% of the sensors have communication points (a sensible situation based mostly on utility expertise), distribution engineers must verify and troubleshoot 50 sensors per day on common. The good sensor communication losses could possibly be attributable to low mobile sign energy, low energy provide, or deliberate or unplanned outages. If it takes 10 minutes to research one sensor, it’s going to trigger the engineering group round 500 minutes per day simply to research the questionable good sensors.

Slightly than checking good sensor data from totally different purposes or techniques to seek out solutions, a sensor standing dashboard solves this downside by aggregating standing statistics throughout all sensors by totally different attributes, together with sensor location, communication standing, and distributions in several areas, substations, and circuits.

Within the following sensor standing dashboard, a hypothetical utility has 102 good sensors (every location wants three sensors for phases A, B, and C) deployed in 5 substations and 6 circuits. Throughout regular operations, good sensor studies load information each 5–quarter-hour, and the occasion information (totally different fault occasions) might come at any time relying on the circuit scenario.

A number of panes are designed to assist distribution engineers reply vital questions on good sensors and facilitate troubleshooting in case communication points occur to good sensors:

  • Abstract – The highest abstract pane supplies a fast look of the good sensor statistics, equivalent to variety of substations, circuits, good sensors with good communications, or good sensors which have communication points.
  • Good Sensor Standing By Location – This pane exhibits the geographical distributions of all of the good sensors. Totally different colours are used to exhibit good sensor operational standing. On this case, 4 of the sensors have communication points, that are proven in pink on the map. The operator can establish the questionable sensors, zoom in, and decide the precise location of those sensors. When operators choose up the questionable good sensors, the geo-map can auto deal with these good sensors as effectively.
  • Sensor Standing By Substation and Circuit – This pane provides operators a look of good sensors by substation and circuit, equivalent to variety of wholesome good sensors and variety of sensors with communication points.
  • Unhealthy Sensor Particulars – This pane supplies details about questionable good sensor information.
  • Mobile Communication Sign Power Distribution – Good sensors transmit information to the cloud utilizing mobile communication. If the sign energy is decrease than -100 dBm to -109 dBm (thought-about poor sign of 1 to 2 bars), the sign may be too weak for the sensor to transmit information. Distribution strains present energy to the good sensors. If the road present is decrease than 5-10 Amps, the sensor could not have sufficient energy to transmit information as effectively. Due to this fact, mobile communication energy and circuit hundreds present vital data for operators to slim down the potential root causes of the good sensor communication loss points. The Mobile Communication Sign Power Distribution pane supplies this data. Pink dots signify good sensors with both very low sign energy or very low circuit load, orange dots present average sign energy and circuit load, and inexperienced dots are the sensors with sturdy sign energy in addition to massive circuit load.
  • Good Sensor Well being Standing Pattern – Though real-time data is necessary to know the good sensors’ standing stay, it’s vital to study the well being pattern of good sensors as effectively. The Good Sensor Well being Standing Pattern pane supplies a sample displaying whether or not the general operations of the good sensor are higher or worse by week or day. Operators can select the time vary, substation, or circuit to study extra granular data.
  • Sensor Distribution by Substation and Sensor Distribution by Circuit – These panes assist the operator study the good sensor deployment distribution data.
  • Good Sensor Checklist – This sensor element pane supplies complete data of the good sensors in a tabular view in case the operator desires to look or kind sensors by element data.

With aggregated good sensor information (geo location, mobile sign energy, distributed circuit energy circulate), operators can rapidly establish problematic sensors and slim down the attainable root causes. This strategy can save a big period of time performing sensor upkeep and troubleshooting—as much as 90% or extra.

In future posts on this sequence, we’ll present you how one can use the paginated studies operate to generate every day studies to enhance the operational effectivity much more. The communication pane additionally exhibits the good sensor distribution utilizing a bar chart, and supplies insights of good sensor deployment data based mostly on area, division, substation, and circuit.

Distribution community occasions dashboard

Good sensors measure and supply the operational data of the distribution networks. This data is vital for operators to know the circuit operating standing and the distribution of various occasions, equivalent to everlasting outages, momentary outages, line disturbance, or voltage sags and swells. QuickSight helps operators rapidly configure totally different views, insights, and calculations on good sensor data.

When an operator specifies a time vary, QuickSight is ready to present good sensor statistics on varied metrics, equivalent to the next:

  • Whole variety of occasions in comparison with a earlier timeframe
  • Distribution of occasions throughout chosen areas, substations, or circuits
  • Distribution of occasions by area, substation, or circuit
  • Distribution of occasions by occasion sort equivalent to everlasting or momentary faults

This data may help operators decide the areas or fault forms of curiosity and research extra detailed data. It may possibly additionally assist operators establish the substations or circuits with essentially the most occasions and take proactive actions to repair any present or hidden points. The pattern data will also be used to validate the gear restore or circuit enhancement works.


Many utilities at present are experiencing elevated integration of distributed power sources (DERs), equivalent to photo voltaic photovoltaic, and energy electronics hundreds equivalent to variable velocity drive and electrical automobile battery chargers. Nonetheless, the prevailing grid wasn’t initially designed to coordinate these DERs, which might trigger hidden points on the prevailing networks. Numerous good sensors are broadly used to watch the distribution networks to enhance grid resiliency and stability.

On this submit, we confirmed how QuickSight may help energy utility distribution community engineers or operators to visualise good sensor standing in actual time and troubleshoot good sensor points. We mentioned out-of-the-box QuickSight options equivalent to its wealthy suite of visualizations, analytical features and calculations, in-memory information engine, and scalability, which is able to tremendously cut back the time, price, and energy of managing massive variety of good sensors and fixing any issues early.

Good sensors are the eyes and ears of utility distribution networks. With QuickSight BI features, operators can rapidly and simply create circuit occasion dashboards; search, kind, filter, and analyze totally different mission-critical occasions; and assist engineers take early motion when sure abnormalities happen on the distribution networks.

Within the following posts on this sequence, we’ll present you how one can use QuickSight to generate every day paginated studies and use superior options equivalent to pure language processing to conduct superior search and analytics features.

In regards to the Authors

Bin Qiu is a International Companion Options Architect specializing in ER&I at AWS. He has greater than 20 years’ expertise within the power and energy industries, designing, main, and constructing totally different good grid initiatives, equivalent to distributed power sources, microgrid, AI/ML implementation for useful resource optimization, IoT good sensor utility for gear predictive upkeep, EV automotive and grid integration, and extra. Bin is captivated with serving to utilities obtain digital and sustainability transformations.

Steve Alexander is a Senior Supervisor, IT Merchandise at PG&E. He leads product groups constructing wildfire prevention and danger mitigation information merchandise. Latest work has been centered on integrating information from varied sources together with climate, asset information, sensors, and dynamic protecting gadgets to enhance situational consciousness and decision-making. Steve has over 20 years of expertise with information techniques and cutting-edge IT analysis and improvement, and is captivated with making use of artistic considering in technical domains.

Karthik Tharmarajan is a Senior Specialist Options Architect for Amazon QuickSight. Karthik has over 15 years of expertise implementing enterprise enterprise intelligence (BI) options and focuses on integration of BI options with enterprise purposes and enabling data-driven choices.

Ranjan Banerji is a Principal Companion Options Architect at AWS centered on the facility and utilities vertical. Ranjan has been at AWS for five years, first on the division of protection (DoD) group serving to the branches of the DoD migrate and/or construct new techniques on AWS guaranteeing safety and compliance necessities and now supporting the facility and utilities group. Ranjan’s experience ranges from server much less structure to safety and compliance for regulated industries. Ranjan has over 25 years of expertise constructing and designing techniques for the DoD, federal companies, power, and monetary trade.

Related Articles


Please enter your comment!
Please enter your name here

Latest Articles