MuleSoft_Misc Flashcards
Where are dataweave functions packaged in Mule?
Mule functions are packaged in Modules
What are the default functions that are imported in DataWeave scripts?
Functions in the core modules are imported automatically in Mule
What are the alternative solutions that can be used for functions with 2 parameters?
#[contains(payload, MAX)] #[payload contains "MAX"]
What is alternative syntax for the following expression?
sizeOf(filter(payload, (value) -> value.age > 30))
sizeOf(payload filter $.age > 30)
How is the following sequence of functions are executed?
flights orderBy $.price filter ($.availableSeats > 30 )
The last function in the chain is executed first filter and followed by orderBy.
Which function is used for removing duplicate data?
distinctBy
Which function is used to filter records?
Filter
Which function is used to order records on the basis of a key?
orderBy
How to add dash in place of space or any other parameter?
using dasherize function
How to call flows from a dataweave expression?
{a: lookup(‘myFlow’, {b: ‘Hello’})}
Inside lookup function first parameter is the flow name and the second parameter is the payload we want to send to the flow.
In Mule 4, All connectors are operation based and not transport based?
True
How Mule 4 connectors are more flexible?
- File connectors can read anywhere in the flow.
- Database connectors can query anywhere in the flow.
- JMS connector can consume anywhere in the flow.
How do we synchronize data from one system to another?
In data synchronization, initially we need to synchronize the initial payload and subsequent times we need to synchronize only new data.
How do we determine which data need to be synced up and when?
In the first sync store the timestamp and in the subsequent times check if the timestamp is greater than the previous timestamp.
What is a timestamp?
The timestamp is often a:-
1- Creation Timestamp
2- Modification Timestamp
3- Record ID
What are the types of watermarking in Mule?
There are 2 types of watermarking in Mule:-
- Automatic
- Manual
Watermarking option is available for which operations?
Automatic watermarking operation is available for the New and Updated file operation for the family of file connectors.
Watermark can be used to ensure if a file is new? What are the other options that can be used to ensure if a file is new?
Delete, move, filter
How does the db query looks like when we enable watermarking?
SELECT * FROM table WHERE TIMESTAMP > :watermark
How does watermark behaves?
On each poll the component will go through all the retrieved rows and store the maximum value obtained.
Handling idempotency across concurrent requests?
A new poll can be executed before the watermark is updated if
- The poll interval is small
- The amounts of rows is big
- Processing one single row takes too much time.
In order to handle the idempotency and avoid reprocessing of single record multiple times what should we use?
Specify an ID column make sure the row is not processed again if:-
- processing of it has already been done.
- or the processing of record is underway.
What are steps involved in manual watermarking?
The general process is:-
- Schedule when a flow should be executed.
- Give the watermark default value.
- on the first sync
- Determine a new watermark value
- store the watermark value so it is available in the future for other flows.
- on the later sync
- Retrieve the watermark from storage.
- Check if each item in the data set should be retrieved based on watermark value.
What are types of scheduling strategies?
- Fixed Frequency
- CRON Job
What is the purpose of Object Store?
We can use object store to determine and store the watermark value
What is JMS?
JMS is a widely-used API for enabling applications to communicate through the exchange of messages.
What are the different messaging models supported by JMS?
- Queues and Topics
What is Queue based communication?
Queue is 1 to 1 approach of communication
What is Topic based communication?
Topic is 1 to many approach of communication
Which is the default message broker used by Mule?
Active MQ broker
What are the different types of message broker?
- Active MQ broker
- Rabbit MQ broker
What is for each scope used for?
for each scope processes the collection of payload in a synchronized fashion
What is the purpose of parallel for each?
parallel for each is exactly similar to for each except apart it processes the records in an Asynchronous fashion.
If we have an ArrayList of LinkedHashMap then inside of for each what it will be?
It will be LinkedHashMap inside the parallel for each scope.
What is the purpose of Batch Processing?
It’s used for processing a huge data set running into millions of records
How does a Batch Job works?
A batch job consists of one or more batch steps which act upon the records as they move through the batch job.
What are the different phases a batch job contains?
In Mule 4 a batch job contains only following phases:-
- Load and dispatch
- process
- on complete
In Mule 3 a batch job contains following phases:-
- Input
- Load and dispatch
- process
- on complete
What are steps involved in Load and Dispatch phase?
Load and dispatch(implicit)
- performs “behind-the-scene” work
- Splits payload into a collection of records
- Creates a persistent queue and stores each record in it.
Process(Required)
- Asynchronous process the records
- Contains one or more batch steps
On Complete(Optional)
- Reports summary of records processed
- Provides insight into which records failed so you can address issues
How record processing works?
- one queue exists
- Each record
- keeps track of what steps it has been processed through.
- Moves through the processors in the first step
- Is sent back to the queue
- Waits to be processed by the second step
- This repeats until each record has passed through every step.
Note: All records do not have finish processing in one step before any of them are sent to the next step.
What is block size in batch processing?
100
What is the thread size?
A thread profile of 16 threads per job is used
Explain threading profile of 16 threads?
- Each of 16 threads processes a block of 100 records.
- Each thread iterates through that block processing each record, and then each block is queued back and then the process continues.
Is the default configuration of batch processing enough for all batch processing records?
Yes default configuration of batch processing is enough for all batch processing records but it is customizable.
How do we handle record-level errors during processing?
For handling failure of records there are 3 options in batch processing:-
- Stop processing the entire batch(default)
- Continue processing the batch
- Continue processing the batch until a max number
How to filter the records in batch processing?
We have 2 attributes to filter records
- An accept expression
- An accept policy
What is batch job?
Batch Job scope (EE only) for complex batch jobs
- created especially for processing data sets.
- Splits messages into individual records and performs action upon each record.
- Can have multiple batch steps and these can have filters
- Record-level data is persisted across steps using variables.
- Can handle record level failures so the job is not aborted.
- The batch job returns an object with the results of the job for insights into which records were processed or failed.
