Individual Assignment

This is the assignment you will do in the first four weeks of the course. In it, you will create a program to process data from smart utilities: gas and electricity meters, and turn this data into a series of plots.

On a high level your program should be able to parse so-called 'telegrams' produced by smart meters. The exact specification of these telegrams is a separate document, you can find here. The parsed data should be converted into some kind of useful data structures (enums, structs, etc) and from that you will generate the data of several plots. This data you can feed into a utility we have provided to you, which will turn the data into actual visualizations for you.

We expect you to work on this assignment gradually while following the lectures of this course. Using the knowledge you've gained from each lecture by applying it to this assignment. That being roughly in the following order:

1. Create data structures matching the format of the telegrams (see the second grading point)
2. Parse strings into these data structures
3. Aggregate data and test the soundness of your program

If you run into any trouble or if anything is unclear, please come to the labs to ask your questions!

Note: There are certain restrictions you have to work with while making this assignment, like not being allowed to use all libraries.

Required Setup

Please read Software Setup to see what packages etc. you need for running the template.

Data Aggregation

The telegrams you parse contain data. Telegrams come in sequences with timestamps. We ask you to implement a simple pipeline that takes these sequences of telegrams, and plots the data contained using the plotting library we provide you called tudelft-dsmr-output-generator. The template you will start with already includes this library.

Using tudelft-dsmr-output-generator we expect you to display:

• Current of each of the three power phases over time (see current_over_time in the library).
• Voltage of each of the three power phases over time (see voltage_over_time in the library).
• A linechart of energy production and consumption over time (see energy_over_time in the library)
  • Each telegram is a datapoint (total energy consumed and total energy produced)
  • The plot should show the difference with the previous datapoint. i.e. even though the data is cumulative, the plot should not be.
  • Always use the first telegram as a reference. Discard the data in the first telegram, but plot the difference with the second telegram, and then the third, etc.
  • Only plot the last 12 datapoints.
• A gas over time chart that works similarly to energy production in that the reporting is cumulative but we expect you to plot the differences
  • However, there is no concept of gas production.
• An overview of events that occurred in the sequence of telegrams
  • Ordered by date!

IMPORTANT NOTE: Time is hard! To help you, there is a function exposed in tudelft-dsmr-output-generator called date_to_timestamp. If you parse dates correctly and use this function, you don't need to worry about timezones at all.

Graphs

Below you can see the expected output of should_parse_4_recursive.dsmr, which you got in your examples directory.

And here is the expected output of should_parse_3.dsmr (which is non-recursive)

Below you can see in detail the graphs from the expected output of should_parse_4_recursive.dsmr.

Gas graph

Current graph

Voltage graph

Energy graph

JSON Example

The following is a snippet from the generated JSON file that is expected using the should_parse_4_recursive.dsmr.

{
  "data": [
    {
      "produced": 15.0,
      "consumed": 8.0,
      "timestamp": 1072916922
    },
    {
      "produced": 17.0,
      "consumed": 10.0,
      "timestamp": 1072916982
    },

    // ...
  ]
}

Grading

Your grade starts at a 1. For each requirement you implement you get the specified amount of points below. Up to a maximum grade of a 10. You will see that in the rubric below, you can receive up to 9 points reflecting this.

You need to have a minimum of a 5.0 for this assignment to be allowed to follow the second part of this course (the group project) and thus be able to pass this course.

• Implement the complete parsing of v1 of the protocol (2pts).
  • You have to use structs, enums and at least one match statement
  • note: if unable to parse certain constructs, partial points may be awarded
• A sensible data format that represents the structure of telegrams. This item will be manually graded based to up to (1pt). With "sensible", we mean:
  • You use enums to represent parts of the format which make sense to be enums.
  • You use structs to represent parts of the format which make sense to be structs.
  • Certain telegram structures which are never valid (hint: telgrams containing different kinds of information at the same time) cannot be represented by your format.
• Reject invalid telegrams (1pts)
  • This requires you to reject invalid telegrams of all the protocol versions you implement
  • On rejection: exit gracefully with exit code 42.
  • note: partial points may be awarded
• Output of the aggregated data format as specified in the assignment above (1pts)
  • Make sure to also do this for v1.2 of the protocol when you implement it.
  • With this we mean the plotting of the data using the tudelft-dsmr-output-generator.
• Unit testing of components of your program (0.75pts)
  • A few basic smoke tests (0.15pts)
  • Substantial testing of at least 10 distinct unit tests covering 40%¹ of your code (0.15pts)
  • Thorough testing with unit and integration tests with in total least 25 distinct tests covering 80%¹ of your code (0.3pts)
  • Unit and smoke testing as defined in Lecture 7 (0.15 points)
  • note: no further partial points can be obtained
• Error Handling (0.75pts)
  • tip: use .expect(..) in your code instead of simply .unwrap() before attempting this so that it is easier to implement when you do get around to it.
  • No panics given malformed data
  • No panics on malformed unicode input
  • On errors, the program outputs a message and nonzero exit status but does not panic.
  • note: partial points may be awarded
• Integrated Error Handling with Testing (0.5pts)
  • Different erroneous inputs return different kinds errors
  • Have at least 10 different error cases across your program
  • Tests verify that your parser rejects erroneous inputs for the right reason
• Implement version 1.2 of the protocol (parsing and aggregation) (2pts)
  • Recursively parse sub-telegrams
  • When graphing data, information should be accumulated (added together) from sub-telegrams (so if there is a toplevel electricity meter and a sub-electricity meter the total energy consumed and produced should be combined (usually by summing)). The Date of sub-telegrams does not matter, assume all data is produced at the same time.
  • Parse Gas information when the g extension is found
  • note: partial points may be awarded

Progress Expectation

Below is a recommendation of where you should be in the project for each of the 4 weeks.

Ideally, implementation of your tests should happen parallel to your development of the actual application. Just make sure that the first time your code compiles isn't in week 4, and test that small parts work starting in week 1.

Limitations

For this assignments you are not allowed to:

• Exchange code with other students. We'd like to evaluate how good you are at programming, not your friends. Discussing concepts during labs is strongly encouraged, but make sure you show what you can do.
• You are not allowed any additional dependencies, except those already present in the template code you received. That means that you are limited to the following crates:
  • thiserror (Or other purely error-handling libraries. If you choose any other, discuss with a TA)
  • itertools (for convenience, you don't need this crate)
  • tudelft-dsmr-output-generator (our library)

If you get stuck

• Look at what the compiler is telling you! It's often mighty useful
• You can always ask a TA during one of the labs which run twice a week
• cargo clippy can tell you things about your code the compiler wouldn't even catch

Hand-In Details

The deadline of the assignment is Sunday the 29th of September, at 23.59, and we will only check the latest commit on the main branch before this date.

Learning Goals

• Gain confidence in your ability to write a small but useful Rust program, specifically:
  • use control flow primitives and use match
  • design data types to match data
  • work with string data
  • using iterators
  • using recursion
  • writing tests
  • handle errors and validate input
• Learn how to read requirements and implement them
• Learn how to write robust programs by writing (unit) tests and applying proper error handling
• Gain some elementary knowledge about using the Git VCS