tag:blogger.com,1999:blog-68443141779476327262024-02-20T01:42:10.774-08:00Data - Operation - Integration and GlueBlablahttp://www.blogger.com/profile/06539104056501152848noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-6844314177947632726.post-41144850906601055472015-07-19T23:14:00.001-07:002015-07-20T04:25:06.368-07:00IODA By Ralf Westphal<div dir="ltr">
Ralf Westphal has now written an english blog about Integration/Operation/Data/API short named as IODA:</div>
<div dir="ltr">
<a href="http://geekswithblogs.net/theArchitectsNapkin/archive/2015/04/29/the-ioda-architecture.aspx">http://geekswithblogs.net/theArchitectsNapkin/archive/2015/04/29/the-ioda-architecture.aspx</a></div>
<div dir="ltr">
This is exact the same pattern than mine described here, but a little more general description.</div>
<div dir="ltr">
One very important point is, that IODA does say nothing about how to bundle/implement Integration, Operation and Data.</div>
<div dir="ltr">
It is allowed to use Integration and Operation methods and Data inside of a single class, that may be seen as a Data class or an Operation class from outside. It depends on what a class is using seen from outside as a black box.</div>
Blablahttp://www.blogger.com/profile/06539104056501152848noreply@blogger.com0tag:blogger.com,1999:blog-6844314177947632726.post-56734860495634209692015-01-13T21:55:00.001-08:002015-01-21T12:23:50.767-08:00First example: Validating birth dates in database<div dir="ltr">
In the first example we have to implement a validation of a birth date, stored in table "person", column "birthdate".<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
Many times I have seen implementations like the following pseudocode:<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;">validate_birthdate () {<br>
string today = today_iso();<br>
foreach (Person person = select ("select id, surname, prename,</span><br>
<span style="font-family: Courier New, Courier, monospace;"> birth, valid_state from person where...")</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> ) {<br>
bool update = false;<br>
if (regex.match(person.date, </span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> "^\d{4,4}-[01]\d-[0-3]\d$")</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> ) {<br>
if (person.date > today) {<br>
update = true;<br>
person.valid_state = "invalid birth";<br>
}<br>
}<br>
else {<br>
update = true;<br>
person.valid_state = "syntax birth";<br>
}<br>
if (update) {<br> </span><span style="font-family: 'Courier New', Courier, monospace;">update("update person "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "set valid_state = " + </span><span style="font-family: 'Courier New', Courier, monospace;">person.valid_state + " "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "where id = " + person.id</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> );</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;">
}<br>
}<br>
} </span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
<h3>
What's wrong with this approach?</h3><div><br></div>
It is running and seems to be correct.<br>
</div>
<div dir="ltr">
But how to test this code?</div><div dir="ltr"><br></div>
<div dir="ltr">
You need a test database.<br>
You have to add rows into the person table. Possibly other rows have to be added before.<br>
Read in rows added.<br>
Then run method for test.<br>
Read again rows and compare with original rows.<br>
Delete added rows after test.<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
Possibly your SW has unexpected side effects. Or expected side effects, i.e. counters, ids, ...<br>
Possibly your SW destroys something in the database.<br>
Or not all of your testrows will/can be deleted.<br>
Possibly you run in conflict with other tests using the same tables.<br>
Or other tests causing problems in you tests somehow.<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
Best would be a private database, but creation may need hours or days. And you need space on disk, permissions, and... </div><div dir="ltr"><br></div>
<div dir="ltr">
And, also very important, it is a lot of work if you use mocking and testdoubles and write/use a database mock instead of a real database.</div><div dir="ltr"><br></div>
<div dir="ltr">
<h3>
What todo?</h3><div><br></div>
</div>
<div dir="ltr">
First separate I/O from operation, extract validation into method do_validate_birth(person). Very easy with Eclipse and Visual Studio, they will do it for you. Just select body and choose "extract method" or something similary:</div>
<div dir="ltr">
<br></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;">validate_birthdate () {<br>
foreach (Person person = select ("select id, surname, prename,</span><br>
<span style="font-family: Courier New, Courier, monospace;"> birth, valid_state from person where...")</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> ) {<br>
bool update = this.do_validate_birth(person);<br>
if (update) {<br> </span><span style="font-family: 'Courier New', Courier, monospace;">update("update person "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "set valid_state = " + </span><span style="font-family: 'Courier New', Courier, monospace;">person.valid_state + " "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "where id = " + person.id</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> );</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;">
}<br>
}</span><br>
</div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;">}</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;">bool do_validate_birth(Person person) {<br>
bool invalid = false;<br>
string today = today_iso();<br>
if (regex.match(person.date, "^\d{4,4}-[01]\d-[0-3]\d$")) {<br>
if (person.date > today) {<br>
invalid = true;<br>
person.valid_state = "invalid birth";<br>
}<br>
}<br>
else {<br>
invalid = true;<br>
person.valid_state = "syntax birth";<br>
}<br>
return invalid;<br>
}</span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
What are the benefits?</div>
<div dir="ltr">
See the test:</div>
<div dir="ltr">
<br></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;">test_do_validate_birth() {<br>
object_in_test = new PersonValidator();</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"> Person person_to_validate = new Person;<br>
person.date = "2100-10-11";</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"> // now do the test<br>
bool result_is_invalid </span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> = object_in_test.do_validate_birth(person)</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> ;</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"> // validate test<br>
if (! result_is_invalid) {<br>
print_line </span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> "test of "</span></div><div dir="ltr"><span style="font-family: 'Courier New', Courier, monospace;"> + "object_in_test</span><span style="font-family: 'Courier New', Courier, monospace;">.do_validate_birth(person) "</span></div><div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"> + "failed!"</span></div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"> ;<br>
}<br>
}</span><br>
</div><div dir="ltr"><span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
All problems described above are gone! Totally! And you do no longer need mocks or testdoubles. </div>
<div dir="ltr">
Using a Testframework could reduce effort for writing the tests additionally. In Perl you just code</div>
<div dir="ltr">
<br></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;">ok (! $object_in_test->do_validate_birth($person),</span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"> "! Person->do_validate_birth(person) for '$date' ");</span></div>
<div dir="ltr">
<br></div>
<div dir="ltr">
for doing the test and validation.</div>
<div dir="ltr">
<br></div>
<div dir="ltr">
The Regex.match() call in do_validate_birth() should also be extracted into a rule or method like</div>
<div dir="ltr">
<br></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;">is_iso_date_str(String date_string) {<br>
return regex.match(date_string, "^\d{4,4}-[01]\d-[0-3]\d$");<br>
}</span></div>
<div dir="ltr">
<span style="font-family: Courier New, Courier, monospace;"><br></span></div>
<div dir="ltr">
So the if line<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;">if (regex.match(person.date, "^\d{4,4}-[01]\d-[0-3]\d$")) {</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
is replaced by<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;">if (is_iso_date_str(person.date)) {</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
Now you need no longer knowledge about regexes to see what person.date should be.<br>
And you can use and test is_iso_date_str() elsewhere.<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
But what about validate_birthdate? It still contains logic and a loop.</div>
<div dir="ltr">
<br>
<span style="font-family: "Courier New",Courier,monospace;">validate_birthdate () {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> foreach (Person person = select ("select id, surname, prename, </span><br>
<span style="font-family: "Courier New",Courier,monospace;"> birth, valid_state from person where...")</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"> ) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> bool update = this.do_validate_birth(person);</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> if (update) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> update("update person "</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"> + "set valid_state = " + </span><span style="font-family: "Courier New",Courier,monospace;">person.valid_state + " "</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"> + "where id = " + person.id</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"> );</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br><span style="font-family: "Courier New",Courier,monospace;">}</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
Let's change it to<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;">validate_birthdate () {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> LIST person_list = select_persons();</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> do_validate_list_of_births(person_list);</span><br>
<span style="font-family: "Courier New",Courier,monospace;">}</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
It contains no logic now. select_persons() does the select in the database, it does no longer interest, how.<br>
<br>
do_validate_list_of_births(person_list) - the extracted method - does an update in Database:<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;">bool do_validate_list_of_births(List person_list) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> foreach (Person person in person_list) ) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> bool update = this.do_validate_birth(person);</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> if (update) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;">update("update person "</span><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "set valid_state = " + </span><span style="font-family: 'Courier New', Courier, monospace;">person.valid_state + " "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "where id = " + person.id</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> );</span></div>
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br>
<span style="font-family: "Courier New",Courier,monospace;">}</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
To handle this, we need an Action class or a closure:<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;">bool do_validate_list_of_births(List person_list,</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"> Action update_action</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"> ) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> foreach (Person person in person_list) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> bool update = this.do_validate_birth(person);</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> update_action(person, update);</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br>
<span style="font-family: "Courier New",Courier,monospace;">}</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
Now only a loop is left. Trivial to test, just give action doing nothing for test.<br>
<br>
Action may be created outside or inside of validate_birthdate:<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;">validate_birthdate () {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> LIST person_list = this.select_persons();</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> do_validate_list_of_births(person_list, \update_action());</span><br>
<span style="font-family: "Courier New",Courier,monospace;">}</span><br>
</div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
\update_action() is a reference to function update_action and defined like:</div>
<div dir="ltr">
<br>
<span style="font-family: "Courier New",Courier,monospace;">update_action(Person person, bool update) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> if (update) {</span><br>
<span style="font-family: "Courier New",Courier,monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;">update("update person "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "set valid_state = " + </span><span style="font-family: 'Courier New', Courier, monospace;">person.valid_state + " "</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> + "where id = " + person.id</span></div><div dir="ltr" style="font-family: sans-serif;"><span style="font-family: 'Courier New', Courier, monospace;"> );</span></div><div dir="ltr">
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br>
<span style="font-family: "Courier New",Courier,monospace;">}</span></div><div dir="ltr"><span style="font-family: "Courier New",Courier,monospace;"><br></span></div>
<div dir="ltr">
<h3>
Summary </h3><div><br></div>
</div>
<div dir="ltr">
We have now 6 methods instead of 1:</div><div dir="ltr"><br></div>
<div dir="ltr">
<h4>
--- Integration, no logic, no test needed</h4><div><br></div>
</div>
<div dir="ltr">
this.validate_birthdate()<br>
do_validate_list_of_births()</div><div dir="ltr"><br></div>
<div dir="ltr">
<h4>
--- Operations doing the validation</h4><div><br></div>
</div>
<div dir="ltr">
do_validate_birth(person)<br>
is_iso_date_str(date_string)</div><div dir="ltr"><br></div>
<div dir="ltr">
<h4>
--- Operations for I/O </h4><div><br></div>
</div>
<div dir="ltr">
update_action(person, update)<br>
select_persons()</div><div dir="ltr"><br>
<h3>
Benefits</h3><div><br></div>
</div>
<div dir="ltr">
But now the code is easy to understand, easy, fast to test -> less/no errors and easy to reuse.<br>
And as result easy to adopt/change.<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
Only validate_birthdate( ) has to stay in original class. All other methods may be moved to other classes. And the call from outside to validate_birthdate() stays unchanged.<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
So we redesigned a part of code without effects to outside.<br>
</div><div dir="ltr"><br></div>
<div dir="ltr">
With a little bit of praxis you split it like this during development without much more development effort, but much less error solving effort afterwards.</div>
Blablahttp://www.blogger.com/profile/06539104056501152848noreply@blogger.com0tag:blogger.com,1999:blog-6844314177947632726.post-43804005631377082142015-01-08T03:44:00.000-08:002015-01-12T09:07:20.884-08:00Data - Operation - Integration and Glue<h2>
Data - Operation - Integration and Glue</h2>
<h3>Trying to build up software by atom components like molecules</h3>
<div>
<br></div><div>Hi all,</div><div><br></div><div>one year ago I started separation of my classes into data, operation, input, output, integration and glue classes. This was inspired by blog of Ralf Westphal:</div>
<div>
<br></div>
<div>
http://blog.ralfw.de/2013/04/software-fraktal-funktionale.html</div><div><br></div><div>If you follow this approach, you don't need any mocking or testdoubles anymore. Yes really!!</div><div>And then you can easily write tests.</div>
<div>
<br></div><div>I think, this may be a new programming paradigm, I call it "molecular software development".</div><div><br></div><div>But how to illustrate this?</div><div><br></div><div>Think of classes like atoms, then a SW component will be a molecule.</div>
<div>
<br></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">D: Data (Daten)</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">C<span style="font-family: 'Courier New', Courier, monospace;">: Converte<span style="font-family: 'Courier New', Courier, monospace;">r</span></span> </span><br>
<span style="font-family: Courier New, Courier, monospace;">O: Operation</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">I: Integrator</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">B: Binder, Glue (Verbinder)</span></div><div><span style="font-family: 'Courier New', Courier, monospace;">F: Factory</span></div><div><span style="font-family: 'Courier New', Courier, monospace;"><br></span></div><div><span style="font-family: sans-serif;">Image missing....</span><span style="font-family: 'Courier New', Courier, monospace;"><br></span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">Data classes are full egocentric, they only can be connected with other data. They have no complex functionality, also validation is done in separate operation classes. They will be passed through the system.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">Operations know only themselves and data objects. They have inputs (methods) and cannot directly call other operations or integrations. To call something outside, they have to call glue connector classes (lambdas, if existing in programming language). Here all the logic will be implemented. Only here! Here tests are needed. Many tests, but easy to implement with no need for mocking or testdoubles!</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">Converter are special operations that transform data between different classes or versions. They transform i.e. car.v1 into car.v2 and create, move or change attributes to do that. So all other operations use only a fix version of data, i.e. op1 use car.v1, op2 car.v2.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">Input and Output are separated into own operation classes, because they are difficult to test.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">Integrators and glue connectors should not contain complex logic, no logic is best case. They just convert data and call operations or other integrators. They usually need not to be tested, but they can. Glue is given to operator classes to call something in outside.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">Factory or builder classes are creating instances and combine them to components or applications.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">The idea of "</span><span style="font-family: sans-serif;">molecular software development" is now not to allow other connections than described above. In best case it would be impossible to violate these rules by development environment.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">In very small projects everything may be mixed in a single class, but methods should NOT mix behavior, so you will get operation methods, integration methods, input and output methods.</span></div><div><span style="font-family: sans-serif;"><br></span></div><div><span style="font-family: sans-serif;">To be continued...</span></div>
Blablahttp://www.blogger.com/profile/06539104056501152848noreply@blogger.com0