makehistory can generate duplicate lines was:[Re: ovsqlite - bug?]
Kamil Jońca
kjonca at op.pl
Mon Nov 21 07:51:07 UTC 2022
Kamil Jońca <kjonca at op.pl> writes:
[...]
>
> So the first thigs I should do is to make history consistent with spool
> and then test further.
> But for now I am pretty sure, that sqlite worked properly as, it forces
> uniqueness of (group,artnum ) in overview (and other methods not) -
> and this causes error messages.
> Sorry for bothering you.
>
I tried to rebuild history:
--8<---------------cut here---------------start------------->8---
kjonca at alfa:/var/lib/news%sudo -u news env PATH=$PATH:/usr/lib/news/bin makehistory -f history.n
makehistory: dbzstore duplicate message-id history.n:[2FFF2E79E03CA12C3E3C4F7E398EBAF7]@60072283 Success
makehistory: dbzstore duplicate message-id history.n:[C1067871BE1F25B369114E7C94635D22]@121782076 Success
makehistory: dbzstore duplicate message-id history.n:[6282400F8A293D047E9052468792AF18]@176267656 Success
makehistory: dbzstore duplicate message-id history.n:[65B607B90549808E3D4A4E3E33DD6D4A]@226791333 Success
makehistory: dbzstore duplicate message-id history.n:[E55B9C6F2E865C431F4466F1085D7596]@352606103 Success
makehistory: dbzstore duplicate message-id history.n:[28464C771BBE6A7D471737C28E6C7F6A]@411231076 Success
makehistory: dbzstore duplicate message-id history.n:[1F375DE27593F75A6022530840232809]@470885597 Success
--8<---------------cut here---------------end--------------->8---
You can see that there is 7 messages about duplicates ... (anyone surprised?)
I took a look in new history file and end with command:
--8<---------------cut here---------------start------------->8---
kjonca at alfa:/var/lib/news%grep -n -A2 -B2 '2FFF2E79E03CA12C3E3C4F7E398EBAF7\|C1067871BE1F25B369114E7C94635D22\|6282400F8A293D047E9052468792AF18\|65B607B90549808E3D4A4E3E33DD6D4A\|E55B9C6F2E865C431F4466F1085D7596\|28464C771BBE6A7D471737C28E6C7F6A\|1F375DE27593F75A6022530840232809' history.n
1:[2FFF2E79E03CA12C3E3C4F7E398EBAF7] 1643424642~-~1538492029 @03FF30000000000000000000004400000008@
2-[50AFBD2AD3C16FA233D6E88D7D5B600C] 1643424642~-~1538492036 @03FF30000000000000000000005E00000007@
3-[4CA26D1C22BC1FCB52DCAA5B56F36666] 1643424642~-~1538492212 @03FF30000000000000000000005F00000007@
--
612979-[11F1E04DF20F6533C381DB6C649B374D] 1643427583~-~1566810064 @03FF3000000000000000001FFFE500000007@
612980-[D121941BCEA5E037523AE35801A53596] 1643427583~-~1566810077 @03FF3000000000000000001FFFE800000007@
612981:[2FFF2E79E03CA12C3E3C4F7E398EBAF7] 1643424642~-~1538492029 @03FF30000000000000000000004400000008@
612982:[C1067871BE1F25B369114E7C94635D22] 1643427583~-~1566810112 @03FF31000000000000000000004400000008@
612983-[82BFD6304D1012333F22C559B8AF46A3] 1643427583~-~1566810146 @03FF31000000000000000000009100000007@
612984-[B2EAA6504DB45DC7F33C214880D99A50] 1643427583~-~1566810185 @03FF31000000000000000000009200000007@
--
1242667-[4F893D038B278138E8AFB8E1C899BFFA] 1643431380~-~1595582940 @03FF3100000000000000001FFFF400000007@
1242668-[B22F99E507F22F30250D4A2D28742A19] 1643431380~-~1595582974 @03FF3100000000000000001FFFFD00000007@
1242669:[C1067871BE1F25B369114E7C94635D22] 1643427583~-~1566810112 @03FF31000000000000000000004400000008@
1242670:[6282400F8A293D047E9052468792AF18] 1643431380~-~1595583001 @03FF32000000000000000000004400000008@
1242671-[A0615E54374BB1263836C289CAB75EF9] 1643431380~-~1595583061 @03FF32000000000000000000004600000007@
1242672-[C07606242B08C0D7FC987945FE0CECA1] 1643431380~-~1595583073 @03FF32000000000000000000004700000007@
--
1798639-[BE079E8E4CA8D10A0ED4C8761C04EE9B] 1643433853~-~1623959587 @03FF3200000000000000001FFFF900000007@
1798640-[C52CD86396DB88B01887D56389ECC6E7] 1643433853~-~1623959629 @03FF3200000000000000001FFFFC00000007@
1798641:[6282400F8A293D047E9052468792AF18] 1643431380~-~1595583001 @03FF32000000000000000000004400000008@
1798642:[65B607B90549808E3D4A4E3E33DD6D4A] 1643433853~-~1623959630 @03FF33000000000000000000004400000008@
1798643-[7D4F42A4C19BC04239EBB1C1538E64AE] 1643433853~-~1623959631 @03FF33000000000000000000004800000007@
1798644-[F49DEA0F8095B1B96440EF800427A797] 1643433853~-~1623959632 @03FF33000000000000000000004B00000007@
--
2314183-[BC90903F831BFB1DE8CA39C98F8ED2C9] 1653058806~-~1653058751 @03FF3300000000000000001FFFF600000007@
2314184-[A505379E708A59F16DAA62541CCFDB67] 1653058866~-~1653058823 @03FF3300000000000000001FFFFE00000007@
2314185:[65B607B90549808E3D4A4E3E33DD6D4A] 1643433853~-~1623959630 @03FF33000000000000000000004400000008@
2314186-[1AA051033A3D96E8E79F6F42C2159419] 1643409332~-~1454972341 @03FF3400000000000000000EE11400000006@
2314187-[8FCA5D33F8688E4E75445850D7CCE398] 1643409332~-~1454972378 @03FF3400000000000000000EE11600000006@
--
2921682-[8735D546AC02468CEC974116E2291BBA] 1669003873~-~1669003801 @03FF3400000000000000000EE10E00000007@
2921683-[4A8E1E3096F262C460C7963971964912] 1669004073~-~1669003319 @03FF3400000000000000000EE11000000007@
2921684:[E55B9C6F2E865C431F4466F1085D7596] 1643412102~-~1465897595 @03FF35000000000000000000004400000007@
2921685-[00F1642A04FEAA3F23269EAF258BDC4A] 1643412102~-~1465897596 @03FF35000000000000000000004800000006@
2921686-[9025840433BA049A341F2D7BF71F44AE] 1643412102~-~1465897596 @03FF35000000000000000000004A00000006@
--
3597988-[D8DB3B14DB0DD47F9344179C17974EF7] 1643417459~-~1487957728 @03FF3500000000000000001FFFCF00000006@
3597989-[EB84472A4704BB56196FAE8D119B6102] 1643417459~-~1487957745 @03FF3500000000000000001FFFD100000006@
3597990:[E55B9C6F2E865C431F4466F1085D7596] 1643412102~-~1465897595 @03FF35000000000000000000004400000007@
3597991:[28464C771BBE6A7D471737C28E6C7F6A] 1643417459~-~1487957763 @03FF36000000000000000000004400000007@
3597992-[25B7BD351BFC0DCA37C9E8D143C74313] 1643417459~-~1487957771 @03FF36000000000000000000008800000006@
3597993-[A4C478830BCD26EDF9206CF4DC18CE29] 1643417459~-~1487957812 @03FF36000000000000000000008B00000006@
--
4196192-[E6B5FC05ED4C9BDB2DC0452382676E79] 1643421414~-~1512585723 @03FF3600000000000000001FFFD300000006@
4196193-[C2574DC662BA26F96D83668966312137] 1643421414~-~1512585730 @03FF3600000000000000001FFFD500000006@
4196194:[28464C771BBE6A7D471737C28E6C7F6A] 1643417459~-~1487957763 @03FF36000000000000000000004400000007@
4196195:[1F375DE27593F75A6022530840232809] 1643421414~-~1512585730 @03FF37000000000000000000004400000007@
4196196-[37B8A01C3D7A5849FC290DA5D3CF91BB] 1643421414~-~1512585769 @03FF37000000000000000000007900000006@
4196197-[578459C8F2A952BC0782C85CCC270CD5] 1643421414~-~1512585775 @03FF37000000000000000000007B00000006@
--
4804906-[9A0C78E7194A7BBA54EB5CB20962BCAA] 1643424642~-~1538491933 @03FF3700000000000000001FFFE700000006@
4804907-[55DB71AEF999C1BCD62FAE9BECC0ED6C] 1643424642~-~1538492145 @03FF3700000000000000001FFFE800000006@
4804908:[1F375DE27593F75A6022530840232809] 1643421414~-~1512585730 @03FF37000000000000000000004400000007@
4804909-[D712318D392955A986E84DCC54C5CC1D] 1651442596~1657056641~1651440641 @040000626F03000200000000000000000000@
4804910-[99690AB74014DD13A932527753D33814] 1646094796~-~1646093042 @040000621D69000200000000000000000000@
--8<---------------cut here---------------end--------------->8---
This is clear (at least for me) that for each cycbuff file (except this
named "4") makehistory, after scanning whole buffer, it additionally try
to write entry about "first" message in buffer (compare lines 1-612981,
612982-1242669 and so on)
Here is additional information - my cycbuff config:
--8<---------------cut here---------------start------------->8---
cycbuff:0:/var/spool/news/cycbuffs/0:8388608
cycbuff:1:/var/spool/news/cycbuffs/1:8388608
cycbuff:2:/var/spool/news/cycbuffs/2:8388608
cycbuff:3:/var/spool/news/cycbuffs/3:8388608
cycbuff:4:/var/spool/news/cycbuffs/4:8388608
cycbuff:5:/var/spool/news/cycbuffs/5:8388608
cycbuff:6:/var/spool/news/cycbuffs/6:8388608
cycbuff:7:/var/spool/news/cycbuffs/7:8388608
metacycbuff:0:0,1,2,3,4,5,6,7:SEQUENTIAL
--8<---------------cut here---------------end--------------->8---
note the "SEQUENTIAL" line - this means that all buffers except one are
"fully rotated" (if you know what I mean) i.e.:
--8<---------------cut here---------------start------------->8---
sudo -u news /usr/lib/news/bin/cnfsstat
Class 0 for groups matching "*"
Buffer 0, size: 8.00 GBytes, position: 272 kBytes 7.00 cycles
Newest: 2022-01-29 4:39:43, 296 days, 4:06:37 ago
Buffer 1, size: 8.00 GBytes, position: 272 kBytes 7.00 cycles
Newest: 2022-01-29 5:43:00, 296 days, 3:03:20 ago
Buffer 2, size: 8.00 GBytes, position: 272 kBytes 7.00 cycles
Newest: 2022-01-29 6:24:13, 296 days, 2:22:07 ago
Buffer 3, size: 8.00 GBytes, position: 272 kBytes 7.00 cycles
Newest: 2022-05-20 17:01:09, 184 days, 16:45:11 ago
Buffer 4, size: 8.00 GBytes, position: 3.72 GBytes 6.47 cycles
Newest: 2022-11-21 8:45:05, 0 days, 0:01:15 ago
Buffer 5, size: 8.00 GBytes, position: 272 kBytes 6.00 cycles
Newest: 2022-01-29 1:50:59, 296 days, 6:55:21 ago
Buffer 6, size: 8.00 GBytes, position: 272 kBytes 6.00 cycles
Newest: 2022-01-29 2:56:54, 296 days, 5:49:26 ago
Buffer 7, size: 8.00 GBytes, position: 272 kBytes 6.00 cycles
Newest: 2022-01-29 3:50:42, 296 days, 4:55:38 ago
--8<---------------cut here---------------end--------------->8---
(please ignore the dates, this spool was recreated from archive/backup)
So I think problem is with "makehistory" when comes to "fully rotated"
cycbuff.
KJ
--
http://wolnelektury.pl/wesprzyj/teraz/
More information about the inn-workers
mailing list