BIND 10 #2888: msgq unittest failure on OpenBSD

[BIND 10 Development]

#2888: msgq unittest failure on OpenBSD
-------------------------------------+-------------------------------------
            Reporter:  jinmei        |                        Owner:
                Type:  defect        |  jinmei
            Priority:  very high     |                       Status:
           Component:  msgq          |  reviewing
            Keywords:                |                    Milestone:
           Sensitive:  0             |  Sprint-20130423
         Sub-Project:  Core          |                   Resolution:
Estimated Difficulty:  3             |                 CVSS Scoring:
         Total Hours:  0             |              Defect Severity:  N/A
                                     |  Feature Depending on Ticket:
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------

Comment (by jinmei):

 Replying to [comment:6 vorner]:

 > > I'm not convinced that abusing unit tests this way is the best
 > > approach, but as the existing tests already passed the review-merge
 > > process I won't fight against that at this moment.
 >
 > Well, propose a better approach. [...]
 >
 > Do we have any other approach?

 I'm afraid we are talking about different things.  My main concern was
 that it's not good to use system level tests, whichever tools are
 used, lettuce or python unittest framework or whatever, for things
 that are supposed to be tested in unit tests, e.g., whether each
 method works as expected, including all corner cases.  A known bad
 example against this principle is statistics tests, and I believe we
 all know it's bad.

 You seemed to talk about which tool we should use for system tests,
 specifically whether to use lettuce or use the python unittest
 framework.  I don't have a strong opinion on that point.

 This gap also made me realize I might misunderstand the purpose of
 #1927 and msgq_run_test.py: I thought the subject of #1927 was to
 write more detailed unit tests and the run_test.py was an attempt of
 it like the statistics tests do.

 In terms of how to *run* system tests, I have one suggestion: I'd like
 to separate the current system test using from unit tests, preferably
 if we type in 'make check' or something equivalently simply command
 only unit tests will be run.  Unit tests are supposed to be run very
 frequently and should generally be completed very quickly.  System
 tests can often take more time, which makes us tend to run it less
 frequently (although, current pure unit tests of msgq also takes some
 time, probably because it also involves separate processes.)

 > Anyway, I pushed the fix. I believe it should work, because it actually
 tries
 > to connect. I did run the test 1000 times and it didn't fail here (even
 without
 > the wait there, which could increase the chance of race condition). I'll
 also
 > ask for the branch to be put into the build bot. But if anyone had a
 reliable
 > way to reproduce, I won't say no for him trying it.

 The branch itself looks okay.  As this is an urgent care issue I think
 it should be merged now.  I'd leave it to you whether to continue the
 discussion above and possibly do some development work within this
 ticket.

-- 
Ticket URL: <http://bind10.isc.org/ticket/2888#comment:8>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development