Model Building
- The first step in model building is observing the real system and the various interactions and events that take place in the system.
As model development proceeds, new questions will arise and model developers will come back to this step to learn the true behavior of the system.
Ex: Operators, technicians etc might interact with each other offer their views, but only by observing as third party we understand the system. - The second step is the construction of conceptual model - a collection of assumptions on the components and structure of the system. It also includes hypotheses on values input parameters and other data.
- The third step is model translation, this process involves converting the model concept to a computer program written using special software or simulation languages.
Refer Model Building Diagram
Verification Of Models
- The purpose of verification is to assure that the conceptual model is represented accurately in it’s computerized form.
- The conceptual model often involves abstraction or simplification of system or actual operations.
Common sense suggestions for verification process :
- Have the program checked by someone other than the developer.
- Make a flow diagram for every logical decision that the model can take when an event occurs, follow it and verify.
- Have the program print the input parameters at then end to ensure that it hasn’t changed or transformed them.
- Make the program as self documenting as possible.
- Interactive run controller / debugger can be used to monitor the program as it progresses step by step.
- GUIs are recommended for easy verification.
Validation Of Models
- Verification and validation although are conceptually distinct, usually are conducted together by the developer.
- Validation is the process of comparing the simulation model and it’s behavior to the real system and it’s behavior.
- Validation is done by calibration, it is an iterative process of comparing the model to the real system, making adjustments or changes and comparing again until a useful model is obtained.
- The comparison to the real system is carried by variety of tests. Usually tests are either subjective or objective.
Subjective Tests : Involve knowledgeable people about one or more aspects of the system making judgements about the model and it’s output.
Objective Tests : Objective tests always require data on the system’s behavior plus the corresponding data produced by the model.
Refer Calibration Diagram
Naylor-Finger 3 Step Validation Process
- Build a model that has high face validity
- Validate model assumptions
- Compare the model input output transformations with real system’s data.
Face Validity
- The first goal of simulation model is to construct a model that appears reasonable on it’s face value to users and knowledgeable users.
- The users of the model should be involved in it’s construction right from the conceptualization to implementation.
- The user involvement increases the face validity or credibility of the model without which others might not be willing to trust the results of the model.
- Sensitivity analysis can be used to check the model’s face validity.
- The model builder needs users to verify if model behaves as expected when input parameters are changed.
Validating Model Assumptions
- Model assumptions fall into two class :
Structural assumptions - Involve how the system operates, simplification and abstractions of operations.
Data assumptions - These are based on collection of reliable data and statistical analysis of the data. - Bank Example : Consider a customer queuing and servicing facility in a bank.
Structural assumptions :
2.1 Customers waiting in one line versus many lines.
2.2 Customers served based on FCFS vs priority.
Data assumptions :
2.3 Inter-arrival times of customers during busy and slack periods
2.4 Service times for personal and commercial accounts. - The data reliability needs to be verified by domain experts, in above example it can be bank managers.
- Test correlations and goodness of fit for data using chi-square test or KS test.
Compare Input-Output Transformation To Real System’s Data
- The goal of this step is to validate the model’s ability to predict the future behavior.
- The model accepts input parameters and transforms them into output data, it is this output with respect to the input that will be validated.
- This a completely objective test.
- The model should be accurate enough to make good predictions for a wide range of input datasets of interest.
- It can be done using past/historical data that has been reserved for validation or a turing test.
Types Of Simulations
There are two types of simulations with respect to output analysis:
- Terminating simulation:
Runs for some duration of time TE, where E is a specified event that stops the simulation.
Starts at time 0 under well-specified initial conditions.
Ends at the stopping time TE.
Bank example: Opens at 8:30 am (time 0) with no customers present and 8 of the 11 teller working (initial conditions), and closes at 4:30 pm (Time TE = 480 minutes).
The simulation analyst chooses to consider it a terminating system because the object of interest is one day’s operation.
- Non Terminating Simulation
A non terminating simulation is one that executes continuously.
Runs continuously, or at least over a very long period of time.
Examples: assembly lines that shut down infrequently, telephone systems, hospital emergency rooms.
Study the steady-state (long-run) properties of the system, properties that are not influenced by the initial conditions of the model.
Inverse Transform Technique
- The inverse transform technique can be used to sample from exponential, the uniform, weibull and triangle distributions.
- The basic principle is to find the inverse function of F, F^-1 such that F.F^-1 = F^-1.F = I.
- F^-1 denotes the solution of the equation r = F(x) in terms of r, not 1/F. For example, the inverse of y = x is x = y, the inverse of y = 2 x + 1 is x = (y-1)/2 …
