Difference between revisions of "Performance/load performance analysis"
Zengliangjun (talk | contribs) |
Zengliangjun (talk | contribs) |
||
Line 1: | Line 1: | ||
= A group of test data = | = A group of test data = | ||
== Interesting contrast analysis data SAX parser processing == | == Interesting contrast analysis data SAX parser processing == | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''Start OOo writer, read a document which has a 3.9 MB "content.xml" '''</center> | ||
− | + | |- | |
| ODF handler | | ODF handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 33: | Line 34: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">6529</div> | + | | <div align="right">'''6529'''</div> |
− | | <div align="right">6560</div> | + | | <div align="right">'''6560'''</div> |
− | | <div align="right">6475</div> | + | | <div align="right">'''6475'''</div> |
− | | <div align="right">5934</div> | + | | <div align="right">'''5934'''</div> |
− | | <div align="right">6738</div> | + | | <div align="right">'''6738'''</div> |
− | | <div align="right">6447.2</div> | + | | <div align="right">'''''6447.2'''''</div> |
|- | |- | ||
Line 60: | Line 61: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">6453</div> | + | | <div align="right">'''6453'''</div> |
− | | <div align="right">5568</div> | + | | <div align="right">'''5568'''</div> |
− | | <div align="right">6472</div> | + | | <div align="right">'''6472'''</div> |
− | | <div align="right">6434</div> | + | | <div align="right">'''6434'''</div> |
− | | <div align="right">6436</div> | + | | <div align="right">'''6436'''</div> |
− | | <div align="right">6272.6</div> | + | | <div align="right">'''''6272.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler read the sample "content.xml" '''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 100: | Line 102: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">324</div> | + | | <div align="right">'''324'''</div> |
− | | <div align="right">332</div> | + | | <div align="right">'''332'''</div> |
− | | <div align="right">326</div> | + | | <div align="right">'''326'''</div> |
− | | <div align="right">340</div> | + | | <div align="right">'''340'''</div> |
− | | <div align="right">288</div> | + | | <div align="right">'''288'''</div> |
− | | <div align="right">322</div> | + | | <div align="right">'''''322'''''</div> |
|- | |- | ||
Line 127: | Line 129: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">281</div> | + | | <div align="right">'''281'''</div> |
− | | <div align="right">282</div> | + | | <div align="right">'''282'''</div> |
− | | <div align="right">278</div> | + | | <div align="right">'''278'''</div> |
− | | <div align="right">281</div> | + | | <div align="right">'''281'''</div> |
− | | <div align="right">276</div> | + | | <div align="right">'''276'''</div> |
− | | <div align="right">279.6</div> | + | | <div align="right">'''''279.6'''''</div> |
|} | |} | ||
− | The first “start – end” | + | '''The first “start – end” contain load library cost ,the second “start – end” do not contain。''' |
All test result are warm . | All test result are warm . | ||
− | We can find the OOo processing more | + | We can find the OOo processing more cost than empty XdocumentHandler。 |
== Interesting contrast analysis data Dual-core processing == | == Interesting contrast analysis data Dual-core processing == | ||
The test with 912 xml files ( total size 44.2 MB ) | The test with 912 xml files ( total size 44.2 MB ) | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and single thread process these files.'''</center> | ||
− | + | |- | |
| Single thread | | Single thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 176: | Line 179: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3575</div> | + | | <div align="right">'''3575'''</div> |
− | | <div align="right">3520</div> | + | | <div align="right">'''3520'''</div> |
− | | <div align="right">3471</div> | + | | <div align="right">'''3471'''</div> |
− | | <div align="right">3515</div> | + | | <div align="right">'''3515'''</div> |
− | | <div align="right">3463</div> | + | | <div align="right">'''3463'''</div> |
− | | <div align="right">3508.8</div> | + | | <div align="right">'''''3508.8'''''</div> |
|- | |- | ||
Line 203: | Line 206: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3472</div> | + | | <div align="right">'''3472'''</div> |
− | | <div align="right">3407</div> | + | | <div align="right">'''3407'''</div> |
− | | <div align="right">3424</div> | + | | <div align="right">'''3424'''</div> |
− | | <div align="right">3487</div> | + | | <div align="right">'''3487'''</div> |
− | | <div align="right">3405</div> | + | | <div align="right">'''3405'''</div> |
− | | <div align="right">3439</div> | + | | <div align="right">'''''3439'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and one file one thread process these files.'''</center> | ||
− | + | |- | |
| <center>'''one file one thread'''</center> | | <center>'''one file one thread'''</center> | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 243: | Line 247: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2678</div> | + | | <div align="right">'''2678'''</div> |
− | | <div align="right">2737</div> | + | | <div align="right">'''2737'''</div> |
− | | <div align="right">2532</div> | + | | <div align="right">'''2532'''</div> |
− | | <div align="right">2797</div> | + | | <div align="right">'''2797'''</div> |
− | | <div align="right">2100</div> | + | | <div align="right">'''2100'''</div> |
− | | <div align="right">2568.8</div> | + | | <div align="right">'''''2568.8'''''</div> |
|- | |- | ||
Line 270: | Line 274: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2622</div> | + | | <div align="right">'''2622'''</div> |
− | | <div align="right">2707</div> | + | | <div align="right">'''2707'''</div> |
− | | <div align="right">2664</div> | + | | <div align="right">'''2664'''</div> |
− | | <div align="right">2347</div> | + | | <div align="right">'''2347'''</div> |
− | | <div align="right">2683</div> | + | | <div align="right">'''2683'''</div> |
− | | <div align="right">2604.6</div> | + | | <div align="right">'''''2604.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and two thread process these files.'''</center> | ||
− | + | |- | |
| Two thread | | Two thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 310: | Line 315: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2287</div> | + | | <div align="right">'''2287'''</div> |
− | | <div align="right">2303</div> | + | | <div align="right">'''2303'''</div> |
− | | <div align="right">2327</div> | + | | <div align="right">'''2327'''</div> |
− | | <div align="right">2347</div> | + | | <div align="right">'''2347'''</div> |
− | | <div align="right">2381</div> | + | | <div align="right">'''2381'''</div> |
− | | <div align="right">2329</div> | + | | <div align="right">'''''2329'''''</div> |
|- | |- | ||
Line 337: | Line 342: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2254</div> | + | | <div align="right">'''2254'''</div> |
− | | <div align="right">2279</div> | + | | <div align="right">'''2279'''</div> |
− | | <div align="right">2266</div> | + | | <div align="right">'''2266'''</div> |
− | | <div align="right">2262</div> | + | | <div align="right">'''2262'''</div> |
− | | <div align="right">2320</div> | + | | <div align="right">'''2320'''</div> |
− | | <div align="right">2276.2</div> | + | | <div align="right">'''''2276.2'''''</div> |
|} | |} | ||
== Interesting contrast analysis data Mononuclear processing == | == Interesting contrast analysis data Mononuclear processing == | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and single thread process these files.'''</center> | ||
− | + | |- | |
| Single thread | | Single thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 378: | Line 384: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5063</div> | + | | <div align="right">'''5063'''</div> |
− | | <div align="right">5200.2</div> | + | | <div align="right">'''''5200.2'''''</div> |
|- | |- | ||
Line 405: | Line 411: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">4954</div> | + | | <div align="right">'''4954'''</div> |
− | | <div align="right">5172</div> | + | | <div align="right">'''5172'''</div> |
− | | <div align="right">4953</div> | + | | <div align="right">'''4953'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">4968</div> | + | | <div align="right">'''4968'''</div> |
− | | <div align="right">5009.4</div> | + | | <div align="right">'''''5009.4'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and one file one thread process these files.'''</center> | ||
− | + | |- | |
| <center>'''one file one thread'''</center> | | <center>'''one file one thread'''</center> | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 445: | Line 452: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5328</div> | + | | <div align="right">'''5328'''</div> |
− | | <div align="right">5344</div> | + | | <div align="right">'''5344'''</div> |
− | | <div align="right">5375</div> | + | | <div align="right">'''5375'''</div> |
− | | <div align="right">5734</div> | + | | <div align="right">'''5734'''</div> |
− | | <div align="right">5453</div> | + | | <div align="right">'''5453'''</div> |
− | | <div align="right">5446.8</div> | + | | <div align="right">'''''5446.8'''''</div> |
|- | |- | ||
Line 472: | Line 479: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5328</div> | + | | <div align="right">'''5328'''</div> |
− | | <div align="right">5381</div> | + | | <div align="right">'''5381'''</div> |
− | | <div align="right">5281</div> | + | | <div align="right">'''5281'''</div> |
− | | <div align="right">5656</div> | + | | <div align="right">'''5656'''</div> |
− | | <div align="right">5281</div> | + | | <div align="right">'''5281'''</div> |
− | | <div align="right">5385.4</div> | + | | <div align="right">'''''5385.4'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and two thread process these files.'''</center> | ||
− | + | |- | |
| Two thread | | Two thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 512: | Line 520: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5063</div> | + | | <div align="right">'''5063'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5200</div> | + | | <div align="right">'''''5200'''''</div> |
|- | |- | ||
Line 539: | Line 547: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">4968</div> | + | | <div align="right">'''4968'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">4953</div> | + | | <div align="right">'''4953'''</div> |
− | | <div align="right">5172</div> | + | | <div align="right">'''5172'''</div> |
− | | <div align="right">4954</div> | + | | <div align="right">'''4954'''</div> |
− | | <div align="right">5009.4</div> | + | | <div align="right">'''''5009.4'''''</div> |
|} | |} | ||
Line 550: | Line 558: | ||
Through the above late two sets of data we can get the follows conclusion: | Through the above late two sets of data we can get the follows conclusion: | ||
− | When | + | === When Single-core, a task divide into multiple small tasks;these are same cost, processing with single thread, one task one thread and two thread.( one task one thread performance relatively low), === |
− | + | === When Dual-core, a task divide into multiple small tasks;the waste is close to 0.7-0.8 with one task one thread process ,it is close to 0.66 with two threads process, a single thread process as a benchmark. === | |
− | When Dual-core, a task divide into multiple small tasks;the waste is close to 0.7-0.8 with one task one thread process ,it is close to 0.66 with two threads process, a single thread process as a benchmark. | ||
− | |||
− | |||
Through the first group of data we can get a conclusion: | Through the first group of data we can get a conclusion: | ||
− | That possess great proportion cost, OOo process ODF. | + | === That possess great proportion cost, OOo process ODF. === |
− | |||
= How to do = | = How to do = | ||
How can we improve ODF loading performance. | How can we improve ODF loading performance. | ||
− | + | As multi-core cpu, Can we split process “content.xml” to some small tasks. Process with multi-thread , and merger the result. | |
− | + | As Single-core, Can we split process “content.xml” to some small tasks. Asynchronous load document . | |
<nowiki>* Of course, At the ODF parser will not improve performance?( But now we don't discuss ) </nowiki> | <nowiki>* Of course, At the ODF parser will not improve performance?( But now we don't discuss ) </nowiki> | ||
Line 571: | Line 575: | ||
How can we split process “content.xml” to some small tasks: | How can we split process “content.xml” to some small tasks: | ||
− | a) Split “content.xml” content | + | a) Split “content.xml” content . |
− | b) Colloect the result with sax parser, and Split | + | b) Colloect the result with sax parser, and Split the result. |
= Description = | = Description = | ||
Line 602: | Line 606: | ||
− | <center> | + | {| class="prettytable" |
+ | | colspan="7" | <center>A simple application it implement a XdocumentHandler that colloect the SAX parser result , use UCB read thest files</center> | ||
− | + | |- | |
− | + | | <center>Colloect the SAX parser result</center> | |
− | | <center> | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
− | | | ||
| | | | ||
Line 634: | Line 638: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5677</div> | + | | <div align="right">'''5677'''</div> |
− | | <div align="right">5491</div> | + | | <div align="right">'''5491'''</div> |
− | | <div align="right">5181</div> | + | | <div align="right">'''5181'''</div> |
− | | <div align="right">5812</div> | + | | <div align="right">'''5812'''</div> |
− | | <div align="right">5516</div> | + | | <div align="right">'''5516'''</div> |
− | | <div align="right">5695.4</div> | + | | <div align="right">'''''5695.4'''''</div> |
|- | |- | ||
Line 661: | Line 665: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5108</div> | + | | <div align="right">'''5108'''</div> |
− | | <div align="right">5175</div> | + | | <div align="right">'''5175'''</div> |
− | | <div align="right">5097</div> | + | | <div align="right">'''5097'''</div> |
− | | <div align="right">5126</div> | + | | <div align="right">'''5126'''</div> |
− | | <div align="right">5267</div> | + | | <div align="right">'''5267'''</div> |
− | | <div align="right">5154.6</div> | + | | <div align="right">'''''5154.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and single thread process these files.'''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 701: | Line 706: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3773</div> | + | | <div align="right">'''3773'''</div> |
− | | <div align="right">3752</div> | + | | <div align="right">'''3752'''</div> |
− | | <div align="right">3895</div> | + | | <div align="right">'''3895'''</div> |
− | | <div align="right">4030</div> | + | | <div align="right">'''4030'''</div> |
− | | <div align="right">3686</div> | + | | <div align="right">'''3686'''</div> |
− | | <div align="right">3821.2</div> | + | | <div align="right">'''''3821.2'''''</div> |
|- | |- | ||
Line 728: | Line 733: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3542</div> | + | | <div align="right">'''3542'''</div> |
− | | <div align="right">3680</div> | + | | <div align="right">'''3680'''</div> |
− | | <div align="right">3924</div> | + | | <div align="right">'''3924'''</div> |
− | | <div align="right">3732</div> | + | | <div align="right">'''3732'''</div> |
− | | <div align="right">3762</div> | + | | <div align="right">'''3762'''</div> |
− | | <div align="right">3728</div> | + | | <div align="right">'''''3728'''''</div> |
|} | |} | ||
We can see it is less cost that colloect the SAX parser result. So we can split that throught some rules and multi-thread to process . | We can see it is less cost that colloect the SAX parser result. So we can split that throught some rules and multi-thread to process . | ||
+ | |||
+ | == Implement Description == | ||
+ | It has two policies which split processing “content.xml”task to some small tasks: | ||
+ | |||
+ | a) Split “content.xml”to many sub files. | ||
+ | |||
+ | b) Colloect the result(attribute & content ) from sax parser, and split the result to many parts in some rules. | ||
+ | |||
+ | struct State | ||
+ | { | ||
+ | rtl_uString* mp_elementName; // current element tag name can replace to one flag | ||
+ | xml::sax::XAttributeList* mp_attributes; // current element's attributes | ||
+ | rtl_uString* mp_elementContent; // current element's content | ||
+ | |||
+ | sal_uInt32 m_local; // It's position at vector | ||
+ | sal_uInt32 m_distance; // It's distance from parent element | ||
+ | sal_uInt32 m_count; // It's count of all sub elements | ||
+ | }; | ||
+ | |||
+ | <nowiki>typedef std::vector< State* > StateVectors;</nowiki> | ||
+ | |||
+ | From compared above multi-thread data, situation single-core / multi-core increase performance is different. | ||
+ | |||
+ | So I think it can consider asynchronous processing as single-core, it can consider parallel processing as multi-core. | ||
+ | |||
+ | It is 2 policies X 2 situations processing description( Of couse it is my imagines ): | ||
+ | |||
+ | === Single Core + Many Fragments Asynchronism Process === | ||
+ | === [[Image:]]Single Core + split SAX result Asynchronism Process === | ||
+ | [[Image:]] | ||
+ | |||
+ | === Multi Core + Many fragments Parallel Process === | ||
+ | === [[Image:]]Multi-Core + split SAX result Parallel Process === | ||
+ | [[Image:]] | ||
== Key points == | == Key points == | ||
Line 743: | Line 782: | ||
= Appendix = | = Appendix = | ||
== Here is simple test, with no other considerations, just to test the feasibility of the technology. == | == Here is simple test, with no other considerations, just to test the feasibility of the technology. == | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a XdocumentHandler that colloect the SAX parser result , use UCB read thest files'''</center> | ||
− | + | |- | |
− | | <center> | + | | <center>Colloect the SAX parser result</center> |
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 775: | Line 815: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5527</div> | + | | <div align="right">'''5527'''</div> |
− | | <div align="right">5534</div> | + | | <div align="right">'''5534'''</div> |
− | | <div align="right">5521</div> | + | | <div align="right">'''5521'''</div> |
− | | <div align="right">5761</div> | + | | <div align="right">'''5761'''</div> |
− | | <div align="right">5535</div> | + | | <div align="right">'''5535'''</div> |
− | | <div align="right">5575.6</div> | + | | <div align="right">'''''5575.6'''''</div> |
|- | |- | ||
Line 802: | Line 842: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5392</div> | + | | <div align="right">'''5392'''</div> |
− | | <div align="right">5530</div> | + | | <div align="right">'''5530'''</div> |
− | | <div align="right">5518</div> | + | | <div align="right">'''5518'''</div> |
− | | <div align="right">5477</div> | + | | <div align="right">'''5477'''</div> |
− | | <div align="right">5414</div> | + | | <div align="right">'''5414'''</div> |
− | | <div align="right">5466.2</div> | + | | <div align="right">'''''5466.2'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a XdocumentHandler that colloect the SAX parser result , use UCB read thest files, and one thread process one SAX parser result.'''</center> | ||
− | + | |- | |
− | |||
− | |||
− | |||
− | |||
− | | | ||
| | | | ||
+ | | <div align="right">No . 1</div> | ||
+ | | <div align="right">No . 2</div> | ||
+ | | <div align="right">No . 3</div> | ||
+ | | <div align="right">No . 4</div> | ||
+ | | <div align="right">No . 5</div> | ||
| | | | ||
Line 842: | Line 883: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">28291</div> | + | | <div align="right">'''28291'''</div> |
− | | <div align="right">28403</div> | + | | <div align="right">'''28403'''</div> |
− | | <div align="right">28085</div> | + | | <div align="right">'''28085'''</div> |
− | | <div align="right">27805</div> | + | | <div align="right">'''27805'''</div> |
− | | <div align="right">28034</div> | + | | <div align="right">'''28034'''</div> |
− | | <div align="right">28123.6</div> | + | | <div align="right">'''''28123.6'''''</div> |
|- | |- | ||
Line 869: | Line 910: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">27437</div> | + | | <div align="right">'''27437'''</div> |
− | | <div align="right">27610</div> | + | | <div align="right">'''27610'''</div> |
− | | <div align="right">28179</div> | + | | <div align="right">'''28179'''</div> |
− | | <div align="right">28639</div> | + | | <div align="right">'''28639'''</div> |
− | | <div align="right">28484</div> | + | | <div align="right">'''28484'''</div> |
− | | <div align="right">28069.8</div> | + | | <div align="right">'''''28069.8'''''</div> |
|} | |} | ||
== A group of large data test == | == A group of large data test == | ||
=== OOo calc “content.xml” 35.1 MB / 3.8M === | === OOo calc “content.xml” 35.1 MB / 3.8M === | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''OOo reader “content.xml” '''</center> | ||
− | + | |- | |
| ODF handler | | ODF handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 911: | Line 953: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">31661</div> | + | | <div align="right">'''31661'''</div> |
− | | <div align="right">31048</div> | + | | <div align="right">'''31048'''</div> |
− | | <div align="right">30126</div> | + | | <div align="right">'''30126'''</div> |
− | | <div align="right">27885</div> | + | | <div align="right">'''27885'''</div> |
− | | <div align="right">28865</div> | + | | <div align="right">'''28865'''</div> |
− | | <div align="right">29917</div> | + | | <div align="right">'''''29917'''''</div> |
|- | |- | ||
Line 938: | Line 980: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">32088</div> | + | | <div align="right">'''32088'''</div> |
− | | <div align="right">32752</div> | + | | <div align="right">'''32752'''</div> |
− | | <div align="right">30918</div> | + | | <div align="right">'''30918'''</div> |
− | | <div align="right">29477</div> | + | | <div align="right">'''29477'''</div> |
− | | <div align="right">28593</div> | + | | <div align="right">'''28593'''</div> |
− | | <div align="right">30765.6</div> | + | | <div align="right">'''''30765.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''“XdocumentHandler” empty implement reader “content.xml” '''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 978: | Line 1,021: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2066</div> | + | | <div align="right">'''2066'''</div> |
− | | <div align="right">2162</div> | + | | <div align="right">'''2162'''</div> |
− | | <div align="right">2105</div> | + | | <div align="right">'''2105'''</div> |
− | | <div align="right">2091</div> | + | | <div align="right">'''2091'''</div> |
− | | <div align="right">2068</div> | + | | <div align="right">'''2068'''</div> |
− | | <div align="right">2098.4</div> | + | | <div align="right">'''''2098.4'''''</div> |
|- | |- | ||
Line 1,005: | Line 1,048: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">1988</div> | + | | <div align="right">'''1988'''</div> |
− | | <div align="right">2574</div> | + | | <div align="right">'''2574'''</div> |
− | | <div align="right">1988</div> | + | | <div align="right">'''1988'''</div> |
− | | <div align="right">1982</div> | + | | <div align="right">'''1982'''</div> |
− | | <div align="right">2212</div> | + | | <div align="right">'''2212'''</div> |
− | | <div align="right">2148.8</div> | + | | <div align="right">'''''2148.8'''''</div> |
|} | |} | ||
=== OOo writer “content.xml” 62.6 MB / 2.0M === | === OOo writer “content.xml” 62.6 MB / 2.0M === | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''OOo reader “content.xml” '''</center> | ||
− | + | |- | |
| ODF handler | | ODF handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 1,046: | Line 1,090: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">102675</div> | + | | <div align="right">'''102675'''</div> |
− | | <div align="right">102669</div> | + | | <div align="right">'''102669'''</div> |
− | | <div align="right">102757</div> | + | | <div align="right">'''102757'''</div> |
− | | <div align="right">101872</div> | + | | <div align="right">'''101872'''</div> |
− | | <div align="right">101725</div> | + | | <div align="right">'''101725'''</div> |
− | | <div align="right">102339.6</div> | + | | <div align="right">'''''102339.6'''''</div> |
|- | |- | ||
Line 1,073: | Line 1,117: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">103322</div> | + | | <div align="right">'''103322'''</div> |
− | | <div align="right">102849</div> | + | | <div align="right">'''102849'''</div> |
− | | <div align="right">102837</div> | + | | <div align="right">'''102837'''</div> |
− | | <div align="right">102281</div> | + | | <div align="right">'''102281'''</div> |
− | | <div align="right">102314</div> | + | | <div align="right">'''102314'''</div> |
− | | <div align="right">102720.6</div> | + | | <div align="right">'''''102720.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''“XdocumentHandler” empty implement reader “content.xml” '''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 1,113: | Line 1,158: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">4805</div> | + | | <div align="right">'''4805'''</div> |
− | | <div align="right">5536</div> | + | | <div align="right">'''5536'''</div> |
− | | <div align="right">5534</div> | + | | <div align="right">'''5534'''</div> |
− | | <div align="right">5635</div> | + | | <div align="right">'''5635'''</div> |
− | | <div align="right">5443</div> | + | | <div align="right">'''5443'''</div> |
− | | <div align="right">5390.6</div> | + | | <div align="right">'''''5390.6'''''</div> |
|- | |- | ||
Line 1,140: | Line 1,185: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5074</div> | + | | <div align="right">'''5074'''</div> |
− | | <div align="right">6220</div> | + | | <div align="right">'''6220'''</div> |
− | | <div align="right">5734</div> | + | | <div align="right">'''5734'''</div> |
− | | <div align="right">5642</div> | + | | <div align="right">'''5642'''</div> |
− | | <div align="right">5322</div> | + | | <div align="right">'''5322'''</div> |
− | | <div align="right">5598.4</div> | + | | <div align="right">'''''5598.4'''''</div> |
|} | |} | ||
− | |||
− |
Revision as of 02:07, 24 April 2009
A group of test data
Interesting contrast analysis data SAX parser processing
ODF handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 12759
|
11805
|
13233
|
11312
|
13279
|
|
Start | 6230
|
5245
|
6758
|
5378
|
6541
|
|
6529
|
6560
|
6475
|
5934
|
6738
|
6447.2
| |
End | 31305
|
36262
|
30083
|
28452
|
28584
|
|
Start | 24852
|
30694
|
23611
|
22018
|
22148
|
|
6453
|
5568
|
6472
|
6434
|
6436
|
6272.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 354
|
362
|
355
|
370
|
312
|
|
Start | 30
|
30
|
29
|
30
|
24
|
|
324
|
332
|
326
|
340
|
288
|
322
| |
End | 635
|
644
|
633
|
651
|
588
|
|
Start | 354
|
362
|
355
|
370
|
312
|
|
281
|
282
|
278
|
281
|
276
|
279.6
|
The first “start – end” contain load library cost ,the second “start – end” do not contain。
All test result are warm .
We can find the OOo processing more cost than empty XdocumentHandler。
Interesting contrast analysis data Dual-core processing
The test with 912 xml files ( total size 44.2 MB )
Single thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 003604
|
003549
|
003500
|
003538
|
003492
|
|
Start | 000029
|
000029
|
000029
|
000023
|
000029
|
|
3575
|
3520
|
3471
|
3515
|
3463
|
3508.8
| |
End | 007076
|
006957
|
006924
|
007025
|
006897
|
|
Start | 003604
|
003550
|
003500
|
003538
|
003492
|
|
3472
|
3407
|
3424
|
3487
|
3405
|
3439
|
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 002696
|
002763
|
002560
|
002827
|
002133
|
|
Start | 000018
|
000026
|
000028
|
000030
|
000033
|
|
2678
|
2737
|
2532
|
2797
|
2100
|
2568.8
| |
End | 005318
|
005471
|
005224
|
005174
|
004816
|
|
Start | 002696
|
002764
|
002560
|
002827
|
002133
|
|
2622
|
2707
|
2664
|
2347
|
2683
|
2604.6
|
Two thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 002314
|
002332
|
002356
|
002377
|
002411
|
|
Start | 000027
|
000029
|
000029
|
000030
|
000030
|
|
2287
|
2303
|
2327
|
2347
|
2381
|
2329
| |
End | 004568
|
004611
|
004622
|
004639
|
004731
|
|
Start | 002314
|
002332
|
002356
|
002377
|
002411
|
|
2254
|
2279
|
2266
|
2262
|
2320
|
2276.2
|
Interesting contrast analysis data Mononuclear processing
Single thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 005062
|
005531
|
005516
|
005062
|
005110
|
|
Start | 000062
|
000062
|
000047
|
000062
|
000047
|
|
5000
|
5469
|
5469
|
5000
|
5063
|
5200.2
| |
End | 010016
|
010703
|
010469
|
010062
|
010078
|
|
Start | 005062
|
005531
|
005516
|
005062
|
005110
|
|
4954
|
5172
|
4953
|
5000
|
4968
|
5009.4
|
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 005391
|
005391
|
005422
|
005797
|
005516
|
|
Start | 000063
|
000047
|
000047
|
000063
|
000063
|
|
5328
|
5344
|
5375
|
5734
|
5453
|
5446.8
| |
End | 010719
|
010672
|
010703
|
011453
|
010797
|
|
Start | 005391
|
005391
|
005422
|
005797
|
005516
|
|
5328
|
5381
|
5281
|
5656
|
5281
|
5385.4
|
Two thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 005110
|
005062
|
005516
|
005531
|
005062
|
|
Start | 000047
|
000062
|
000047
|
000062
|
000062
|
|
5063
|
5000
|
5469
|
5469
|
5000
|
5200
| |
End | 010078
|
010062
|
010469
|
010703
|
010016
|
|
Start | 005110
|
005062
|
005516
|
005531
|
005062
|
|
4968
|
5000
|
4953
|
5172
|
4954
|
5009.4
|
Overview
Through the above late two sets of data we can get the follows conclusion:
When Single-core, a task divide into multiple small tasks;these are same cost, processing with single thread, one task one thread and two thread.( one task one thread performance relatively low),
When Dual-core, a task divide into multiple small tasks;the waste is close to 0.7-0.8 with one task one thread process ,it is close to 0.66 with two threads process, a single thread process as a benchmark.
Through the first group of data we can get a conclusion:
That possess great proportion cost, OOo process ODF.
How to do
How can we improve ODF loading performance.
As multi-core cpu, Can we split process “content.xml” to some small tasks. Process with multi-thread , and merger the result.
As Single-core, Can we split process “content.xml” to some small tasks. Asynchronous load document .
* Of course, At the ODF parser will not improve performance?( But now we don't discuss )
How can we split process “content.xml” to some small tasks:
a) Split “content.xml” content .
b) Colloect the result with sax parser, and Split the result.
Description
Split “content.xml” content
Change the document structure, complex nodules defined as a reference link to a single file,the main file contains only simple structure.
It will produce the ODF layout :
content.xml complex/fragment1.xml /fragment2.xml /fragment3.xml /fragment4.xml …
And “content.xml” will be :
… <table:table xlink:href= “complex/fragment1.xml” /> …
I think it will produce the same performance optimization when save documentation, this way.
Colloect the result with sax parser, and Split that
From the above test data, SAX process with relatively less cost itself. We can colloect SAX parser result, and split it to multi part.
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 005706
|
005510
|
005200
|
005824
|
005527
|
|
Start | 000029
|
000019
|
000019
|
000012
|
000011
|
|
5677
|
5491
|
5181
|
5812
|
5516
|
5695.4
| |
End | 010814
|
010685
|
010297
|
010950
|
010794
|
|
Start | 005706
|
005510
|
005200
|
005824
|
005527
|
|
5108
|
5175
|
5097
|
5126
|
5267
|
5154.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 003788
|
003763
|
003908
|
004044
|
003702
|
|
Start | 000015
|
000011
|
000013
|
000014
|
000016
|
|
3773
|
3752
|
3895
|
4030
|
3686
|
3821.2
| |
End | 008330
|
007444
|
007836
|
007776
|
007465
|
|
Start | 003788
|
003764
|
003908
|
004044
|
003703
|
|
3542
|
3680
|
3924
|
3732
|
3762
|
3728
|
We can see it is less cost that colloect the SAX parser result. So we can split that throught some rules and multi-thread to process .
Implement Description
It has two policies which split processing “content.xml”task to some small tasks:
a) Split “content.xml”to many sub files.
b) Colloect the result(attribute & content ) from sax parser, and split the result to many parts in some rules.
struct State { rtl_uString* mp_elementName; // current element tag name can replace to one flag xml::sax::XAttributeList* mp_attributes; // current element's attributes rtl_uString* mp_elementContent; // current element's content
sal_uInt32 m_local; // It's position at vector sal_uInt32 m_distance; // It's distance from parent element sal_uInt32 m_count; // It's count of all sub elements };
typedef std::vector< State* > StateVectors;
From compared above multi-thread data, situation single-core / multi-core increase performance is different.
So I think it can consider asynchronous processing as single-core, it can consider parallel processing as multi-core.
It is 2 policies X 2 situations processing description( Of couse it is my imagines ):
Single Core + Many Fragments Asynchronism Process
[[Image:]]Single Core + split SAX result Asynchronism Process
[[Image:]]
Multi Core + Many fragments Parallel Process
[[Image:]]Multi-Core + split SAX result Parallel Process
[[Image:]]
Key points
The collected information segmentation algorithm
Now for OOo process is not very clear, do n't know whether be expected. Waits further research
Appendix
Here is simple test, with no other considerations, just to test the feasibility of the technology.
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 005560
|
005563
|
005551
|
005791
|
005558
|
|
Start | 000033
|
000029
|
000030
|
000030
|
000023
|
|
5527
|
5534
|
5521
|
5761
|
5535
|
5575.6
| |
End | 010952
|
011093
|
011069
|
011268
|
010972
|
|
Start | 005560
|
005563
|
005551
|
005791
|
005558
|
|
5392
|
5530
|
5518
|
5477
|
5414
|
5466.2
|
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 028326
|
028438
|
028115
|
27835
|
28054
|
|
Start | 000035
|
000035
|
000030
|
30
|
20
|
|
28291
|
28403
|
28085
|
27805
|
28034
|
28123.6
| |
End | 055763
|
056048
|
056294
|
056474
|
056538
|
|
Start | 028326
|
028438
|
28115
|
027835
|
028054
|
|
27437
|
27610
|
28179
|
28639
|
28484
|
28069.8
|
A group of large data test
OOo calc “content.xml” 35.1 MB / 3.8M
ODF handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 38220
|
37340
|
37140
|
66316
|
35378
|
|
Start | 6559
|
6292
|
7014
|
38431
|
6513
|
|
31661
|
31048
|
30126
|
27885
|
28865
|
29917
| |
End | 79549
|
82462
|
81397
|
107576
|
873537
|
|
Start | 47461
|
49710
|
50479
|
78099
|
844944
|
|
32088
|
32752
|
30918
|
29477
|
28593
|
30765.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 2082
|
2178
|
2118
|
2106
|
2082
|
|
Start | 16
|
16
|
13
|
15
|
14
|
|
2066
|
2162
|
2105
|
2091
|
2068
|
2098.4
| |
End | 4070
|
4752
|
4106
|
4088
|
4294
|
|
Start | 2082
|
2178
|
2118
|
2106
|
2082
|
|
1988
|
2574
|
1988
|
1982
|
2212
|
2148.8
|
OOo writer “content.xml” 62.6 MB / 2.0M
ODF handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 108876
|
108729
|
108997
|
108901
|
110140
|
|
Start | 6201
|
6060
|
6240
|
7029
|
8415
|
|
102675
|
102669
|
102757
|
101872
|
101725
|
102339.6
| |
End | 229982
|
228416
|
235011
|
234573
|
233124
|
|
Start | 126660
|
125567
|
132174
|
132292
|
130810
|
|
103322
|
102849
|
102837
|
102281
|
102314
|
102720.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 4825
|
5595
|
5546
|
5658
|
6072
|
|
Start | 20
|
59
|
12
|
23
|
629
|
|
4805
|
5536
|
5534
|
5635
|
5443
|
5390.6
| |
End | 9899
|
11815
|
11280
|
11300
|
11394
|
|
Start | 4825
|
5595
|
5546
|
5658
|
6072
|
|
5074
|
6220
|
5734
|
5642
|
5322
|
5598.4
|