Multi-Threading Kernel for Data Integration
Data integration:The final frontier.
If anyone has worked for a ecommerce company or a company that develops ecommerce system for a brick and mortar site, the question always arrises: Who will be the Master of Inventory(MOI)? Ideally you would want the web site to control the inventory that way when you are “out of stock” the web site will be updated automatically and online users won’t get disappointed by ordering a out of stock item.
The problem arises when the Brick and Mortar(B&M) site does want the website to be the MOI. Usually the reason being that the floor still generates most of the sales and they already have a inventory control system in place and do not want to change it. Your options are :
1) Create a outbound xml channel to your database that updates the inventory periodacally. Problem: expensize as you would need a Biztalk server or something of the sort and complex as you will have to define and map the outbound and inbound structure
2) Have the admin mannually update the products in stock.Problem: Inpractacall even if you created a upload page the user would have to spend hours a day entering and verifing the data.
3) Create a multithreading kernel that polls a location and imports the data based on the MOI export file.
Most Inventory control systems have a export functionality, wither it is in XML or some propietary system, which you can use. You simply need to create a kernal that will poll a export location and pull in the data as soon as the Inventory control system spits it out.
Here the outline of what we want to accomplish:
1. Multi-Threading
2. Live feed back
3. A Folder location where the export file will be continously polled.
Program
Start by opening Visual studio and create a Windows application in C#.
Lets start by adding the basic elements that will contrust our kernel:
1. Add a Start and Stop button and a browse button
2. Add a text box of the File Directory to be inserted
3. Add a Multiline TextBox that will display your messages
4. Drop a OpenFileDialog Control
5. Drop a Worker thread
Your form should look something like this
Coding
Double click the browse button and use the OpenFiledialog control which I named fbd to show and grab a file location. This will be the location were we will poll for a export file.
Sample Code
{
fbd.ShowDialog();
txtFolderLocation.Text = fbd.SelectedPath;
}
Next we will look at the start and stop buttonThinking about what we want to do: First the Start and Stop button we be associated with the ThreadWorker component that previously dropped on the form. Second we want to report all messages to the large Text box.
Coding:Lets look a the properties of the Thread are:
The only property that we are conserned with is the WorkerSupportsCancellation. We want to be able to stop the Thread gracefully, so this property needs to be set to true.Now lets look at the events of the Thread:
The only two things we are concerned win are the DoWork and the RunWorkerCompleted Event. The DoWork is is the function that is going to be called when you start the thread and RunWorkedCompleted is the event called when you push the stop button. Double click on these event to create the Events.DoWork The function of this event is to poll the folder selected previously the code is simple:Code:
private void thread1_DoWork(object sender, DoWorkEventArgs e)
{
while (!thread1.CancellationPending)
{
if (File.Exists(path))
{
//do something with the file
}
}
The only think to notice is that we are looping on the thread1.CancellationPending not being set to true. This allows us to Stop the thread by pressing the stop button.When the Stop button is pressed the
private void thread1_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{}
is called. Use this function to do any clean up code and ideally write the log file.Next we will code the Start and Stop Buttons:Code:StartThe code for the start button is pretty simple. Basically we would like to disable the start button enable the stop button and call the thres to start Code: private void btnStart_Click(object sender, EventArgs e)
{
btnStart.Enabled = false;
btnStop.Enabled = true;
thread1.RunWorkerAsync();
}
Now your DoWork event will fire.Writing messages to the text box
If you call the TextBox’s .Text property to send messages you will get the following error:
Cross-thread operation not valid: Control “txtOutInfo” accessed from a thread other than the thread it was created on.
In order to acess the textbpx Text Propety you need to use a delegateCode: private void SetText(string text)
{
if (this.OutInfo.InvokeRequired)
{
// InvokeRequired required compares the thread ID of the calling thread to the thread ID of the creating thread. If these threads are different, it returns true. SetTextCallback stc =
new SetTextCallback(SetText); this.Invoke(d, new object[] text });}
else
{
this.txtOutInfo.Text += text;
this.txtOutInfo.SelectionStart = OutInfo.Text.Length;
OutInfo.ScrollToCaret();
}
}
And you now have messages:
Now that you set up your Kernel you can data migrate.O.
Hello,
Here is my new blog. I started this blog for one reason. To help other programmers and to pass on my knowledge to the world. My goals with this blog is to show how to do programming for real life situations that arise in the real world. I see a lot of post of theoretical issues and a lot of one line answers. I will attempt to post full solutions to real life problems that web programmers and web companies face.
Feel free to email me or comment on my code.
