Roundtable September 2017

roundtable@muug.ca

9 participants
9 discussions

Looking for some AWK magic
by Wyatt Zacharias 24 Sep '17

24 Sep '17

So I've got a storage appliance that I'm trying to generate a growth chart on, but the output format of the statistics is not in a usable format. The output looks like this, with each day of the week in a column, and then 6 rows of data underneath, and then it repeats. -2- -3- -4- -5- -6- -7- -8- 5827.1 6865.6 3551.2 4649.5 4006.6 15803.5 10305.0 199.4 353.4 175.9 200.7 172.5 584.0 554.2 9.9x 4.4x 6.4x 5.9x 6.3x 5.6x 5.3x 3.0x 4.4x 3.2x 3.9x 3.7x 4.9x 3.5x 29.2x 19.4x 20.2x 23.2x 23.2x 27.1x 18.6x 96.6 94.9 95.0 95.7 95.7 96.3 94.6 -23- -24- -25- -26- -27- -28- -29- 2798.2 2235.3 2357.9 4701.3 9074.3 13796.5 11705.7 86.7 74.1 131.6 178.1 449.3 483.5 473.7 11.4x 12.0x 7.0x 8.1x 3.9x 6.3x 6.7x 2.8x 2.5x 2.6x 3.2x 5.2x 4.6x 3.7x 32.3x 30.2x 17.9x 26.4x 20.2x 28.5x 24.7x 96.9 96.7 94.4 96.2 95.0 96.5 96.0 -30- -31- -1- -2- -3- -4- -5- 2798.6 2274.5 2325.8 2472.1 2526.0 13955.3 11224.2 86.7 88.9 145.2 115.2 105.6 497.9 432.4 10.8x 10.1x 7.4x 8.1x 9.8x 6.2x 6.9x 3.0x 2.5x 2.2x 2.7x 2.4x 4.5x 3.8x 32.3x 25.6x 16.0x 21.5x 23.9x 28.0x 26.0x 96.9 96.1 93.8 95.3 95.8 96.4 96.1 Is there a way I can get awk to process 6 rows of each column at a time? -- Wyatt Zacharias

4 3

Re: [RndTbl] Roundtable Digest, Vol 153, Issue 6
by Michael Doob 21 Sep '17

21 Sep '17

On the other hand awk has 1. a variable NR the contains the current line number 2. a % operator that can be used to compute modulo 7 3. a getline command that reads the next line of input That's all you need. Cheers, Michael ------------------------------------------------------------------ Michael Doob Telephone: (204) 474-9796 Department of Mathematics email: Michael.Doob(a)UManitoba.ca University of Manitoba Winnipeg, MB, Canada R3T 2N2 ------------------------------------------------------------------ ________________________________________ From: Roundtable <roundtable-bounces(a)muug.ca> on behalf of roundtable-request(a)muug.ca <roundtable-request(a)muug.ca> Sent: September 21, 2017 1:23:52 PM To: roundtable(a)muug.ca Subject: Roundtable Digest, Vol 153, Issue 6 Send Roundtable mailing list submissions to roundtable(a)muug.ca To subscribe or unsubscribe via the World Wide Web, visit https://muug.ca/mailman/listinfo/roundtable or, via email, send a message with subject or body 'help' to roundtable-request(a)muug.ca You can reach the person managing the list at roundtable-owner(a)muug.ca When replying, please edit your Subject line so it is more specific than "Re: Contents of Roundtable digest..." Today's Topics: 1. Looking for some AWK magic (Wyatt Zacharias) 2. Re: Looking for some AWK magic (Sean Cody) 3. Re: Looking for some AWK magic (John Lange) ---------------------------------------------------------------------- Message: 1 Date: Thu, 21 Sep 2017 12:01:33 -0500 From: Wyatt Zacharias <wyatt(a)magitech.ca> To: Continuation of Round Table discussion <roundtable(a)muug.ca> Subject: [RndTbl] Looking for some AWK magic Message-ID: <CANd2pR2wwERFj2R6_JswyzT91cpnq+RM7KhgC1goP6CEbbJoiQ(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" So I've got a storage appliance that I'm trying to generate a growth chart on, but the output format of the statistics is not in a usable format. The output looks like this, with each day of the week in a column, and then 6 rows of data underneath, and then it repeats. -2- -3- -4- -5- -6- -7- -8- 5827.1 6865.6 3551.2 4649.5 4006.6 15803.5 10305.0 199.4 353.4 175.9 200.7 172.5 584.0 554.2 9.9x 4.4x 6.4x 5.9x 6.3x 5.6x 5.3x 3.0x 4.4x 3.2x 3.9x 3.7x 4.9x 3.5x 29.2x 19.4x 20.2x 23.2x 23.2x 27.1x 18.6x 96.6 94.9 95.0 95.7 95.7 96.3 94.6 -23- -24- -25- -26- -27- -28- -29- 2798.2 2235.3 2357.9 4701.3 9074.3 13796.5 11705.7 86.7 74.1 131.6 178.1 449.3 483.5 473.7 11.4x 12.0x 7.0x 8.1x 3.9x 6.3x 6.7x 2.8x 2.5x 2.6x 3.2x 5.2x 4.6x 3.7x 32.3x 30.2x 17.9x 26.4x 20.2x 28.5x 24.7x 96.9 96.7 94.4 96.2 95.0 96.5 96.0 -30- -31- -1- -2- -3- -4- -5- 2798.6 2274.5 2325.8 2472.1 2526.0 13955.3 11224.2 86.7 88.9 145.2 115.2 105.6 497.9 432.4 10.8x 10.1x 7.4x 8.1x 9.8x 6.2x 6.9x 3.0x 2.5x 2.2x 2.7x 2.4x 4.5x 3.8x 32.3x 25.6x 16.0x 21.5x 23.9x 28.0x 26.0x 96.9 96.1 93.8 95.3 95.8 96.4 96.1 Is there a way I can get awk to process 6 rows of each column at a time? -- Wyatt Zacharias

1 0

an MIT operating systems course on Github
by Grigory Shamov 14 Sep '17

14 Sep '17

Hi, It might be not that new, but just in case. In the Login magazine I won at last meeting¹s raffle there is an article about the subj. The course is opensourced. https://github.com/teachbsd/course Relies on a rather expensive book though, ³The Design and Implementation of the FreeBSD Operating System². -- Grigory Shamov WestGrid Site Lead / HPC Specialist University of Manitoba E2-588 EITC Building, (204) 474-9625

1 0

More about network time...
by Gilbert E. Detillieux 13 Sep '17

13 Sep '17

Wyatt's presentation on NTPD last night prompted some discussion about other NTP implementations. (Chrony was mentioned in the presentation, and systemd-timesyncd was mentioned in discussion.) The latest Raspbian boot image for the Raspberry Pi (based on Debian stretch) no longer installs ntpd by default, but relies on systemd-timesyncd instead. (Presumably Debian stretch does something similar, though that may not be the case...) Of course, ntpd, chrony, and openntpd are all available as alternatives. The good news is that systemd-timesyncd seems to work for the very simple case of a client-only setup, where you just want to get the time reasonably close to some official time source (it only implements an NTP client, not a server), requires no special configuration, and seems to get out of the way if you install any of the more-capable NTP server packages. The bad news is that it doesn't do as good a job of keeping time in sync. It only contacts one of the available NTP servers, and sticks to that one until it's not reachable, in which case it tries the next available one, etc. It will use as time sources one of the available pool servers for the distribution, or (if provided) one of those indicated by the DHCP server. So, for the typical client, it may be fine. Also, it doesn't give you much by way of stats. You can use the "timedatectl" command, to find out if NTP synchronization is enabled or not, but that's about it. You can also use the "systemctl -l status systemd-tymesyncd" command, and look at the log entries showing which time server was used... Sep 09 07:02:29 raspberrypi systemd-timesyncd[215]: Synchronized to time server 206.108.0.134:123 (2.debian.pool.ntp.org). Sep 09 23:33:45 raspberrypi systemd-timesyncd[215]: Timed out waiting for reply from 206.108.0.134:123 (2.debian.pool.ntp.org). Sep 09 23:33:45 raspberrypi systemd-timesyncd[215]: Synchronized to time server 208.81.1.244:123 (2.debian.pool.ntp.org). Sep 10 00:43:48 raspberrypi systemd-timesyncd[215]: Timed out waiting for reply from 208.81.1.244:123 (2.debian.pool.ntp.org). Sep 10 00:43:48 raspberrypi systemd-timesyncd[215]: Synchronized to time server 174.94.155.224:123 (2.debian.pool.ntp.org). No way to check for offset and other such stats, though. (It may not even track that.) Chrony seems to be a quite capable replacement for NTPD, with a configuration file format that looks quite similar (at least for the basic config). There are a couple different commands to get stats... [root@host ~]# chronyc sources 210 Number of sources = 7 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^* ntp1.mydomain.net 3 10 377 280 -274us[ -216us] +/- 71ms ^+ ntp2.mydomain.net 3 10 377 585 -348us[ -290us] +/- 66ms ^+ ntp3.mydomain.net 3 10 377 510 -696us[ -638us] +/- 68ms ^+ james.kirk.hungrycats.org 2 10 377 229 -3138us[-3138us] +/- 69ms ^+ ns541050.ip-144-217-252.n 3 9 377 390 -2763us[-2705us] +/- 102ms ^+ sanction.treborlogic.com 2 10 377 983 +573us[ +630us] +/- 75ms ^+ 209.115.181.107 3 10 377 455 +13ms[ +13ms] +/- 108ms [root@host ~]# chronyc sourcestats 210 Number of sources = 7 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev ============================================================================== ntp1.mydomain.net 39 19 273m -0.000 0.007 -814us 65us ntp2.mydomain.net 16 9 90m -0.002 0.066 -619us 68us ntp3.mydomain.net 27 17 226m -0.010 0.013 -784us 52us james.kirk.hungrycats.org 6 4 86m +0.160 1.290 -3291us 739us ns541050.ip-144-217-252.n 8 6 60m -0.143 0.280 -3050us 162us sanction.treborlogic.com 6 3 86m +0.287 0.623 +614us 302us 209.115.181.107 12 9 189m +0.129 0.121 +13ms 262us [root@hedge ~]# The default config is set up to query servers in the pool for the distro (rhel in the above example, with the last 4 servers shown), but will also include any NTP servers indicated by DHCP (the first 3 servers shown, with bogus host names in this example). I haven't tried OpenNTPD (The OpenBSD NTP daemon), so I don't know how that one looks compared to the classic NTPD. Gilbert -- Manitoba UNIX User Group E-mail: <gedetil(a)muug.ca> c/o Gilbert E. Detillieux Web: http://muug.ca/ University of Manitoba Phone: (204)474-8161 Winnipeg MB CANADA R3T 2N2 Fax: (204)474-7609

1 0

Re: [RndTbl] Roundtable Digest, Vol 153, Issue 3
by Kat 11 Sep '17

11 Sep '17

I don't know about Firefox, but I meant to write about the headless Chrome CLI in the last newsletter. It's been around since v59/60. https://developers.google.com/web/updates/2017/04/headless-chrome On 8 September 2017 at 12:00, <roundtable-request(a)muug.ca> wrote: > Send Roundtable mailing list submissions to > roundtable(a)muug.ca > > To subscribe or unsubscribe via the World Wide Web, visit > https://muug.ca/mailman/listinfo/roundtable > or, via email, send a message with subject or body 'help' to > roundtable-request(a)muug.ca > > You can reach the person managing the list at > roundtable-owner(a)muug.ca > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Roundtable digest..." > > > Today's Topics: > > 1. Firefox remote control / or headless js? MozRepl gone... > (Trevor Cordes) > 2. massive equifax hack (Trevor Cordes) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 8 Sep 2017 00:24:43 -0500 > From: Trevor Cordes <trevor(a)tecnopolis.ca> > To: MUUG RndTbl <roundtable(a)muug.ca> > Subject: [RndTbl] Firefox remote control / or headless js? MozRepl > gone... > Message-ID: <20170908052443.GA10410(a)pog.tecnopolis.ca> > Content-Type: text/plain; charset=us-ascii > > MozRepl, which I use to remotely control Firefox, is officially broken as > of FF55 and the devs say it won't be fixed and can't really work in the > future anyhow as FF ditches XPCOM. That means I can't use > WWW::Mechanize::Firefox anymore (as demo'd (demoed? demod?...) in a MUUG > meeting a couple of years back). > (https://github.com/bard/mozrepl/issues/68#issuecomment-322212759) > > So I'm left looking for alternatives... Anyone else know of / is using any > good progrmamatic browser remote-controllers? Hopefully they'd support FF > and Perl, but I could convert/find workarounds if it supported different > browser/languages. > > Also, beyond that, are there any good libraries /modules for interacting > with web pages "headless", that is, having the dom and js handled all in > memory without any actual browser or window showing? > > I'd love to hear about personal experience / anecdotes, but just rattling > off some names to research would be helpful too. Thanks! > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Sep 2017 05:06:56 -0500 > From: Trevor Cordes <trevor(a)tecnopolis.ca> > To: MUUG RndTbl <roundtable(a)muug.ca> > Subject: [RndTbl] massive equifax hack > Message-ID: <20170908100656.GA3883(a)pog.tecnopolis.ca> > Content-Type: text/plain; charset=us-ascii > > Equifax blew it and allowed SQL injection (that's soooo 2005) vulns to > leak access to id/credit details of 143M Americans, as well as undisclosed > number of Canadians and UK. This is pretty big news. No word yet on how > Canadians can check their status or access the freebies they are handing > out. > > https://www.theinquirer.net/inquirer/news/3016920/equifax- > hack-exposes-personal-details-of-44-per-cent-of-us-citizens > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Roundtable mailing list > Roundtable(a)muug.ca > https://muug.ca/mailman/listinfo/roundtable > > > ------------------------------ > > End of Roundtable Digest, Vol 153, Issue 3 > ****************************************** > -- ------------------------------------------------------------ Katherine Scrupa Network Technology CCNA, Hons.

1 0

massive equifax hack
by Trevor Cordes 09 Sep '17

09 Sep '17

Equifax blew it and allowed SQL injection (that's soooo 2005) vulns to leak access to id/credit details of 143M Americans, as well as undisclosed number of Canadians and UK. This is pretty big news. No word yet on how Canadians can check their status or access the freebies they are handing out. https://www.theinquirer.net/inquirer/news/3016920/equifax-hack-exposes-pers…

2 1

Firefox remote control / or headless js? MozRepl gone...
by Trevor Cordes 08 Sep '17

08 Sep '17

MozRepl, which I use to remotely control Firefox, is officially broken as of FF55 and the devs say it won't be fixed and can't really work in the future anyhow as FF ditches XPCOM. That means I can't use WWW::Mechanize::Firefox anymore (as demo'd (demoed? demod?...) in a MUUG meeting a couple of years back). (https://github.com/bard/mozrepl/issues/68#issuecomment-322212759) So I'm left looking for alternatives... Anyone else know of / is using any good progrmamatic browser remote-controllers? Hopefully they'd support FF and Perl, but I could convert/find workarounds if it supported different browser/languages. Also, beyond that, are there any good libraries /modules for interacting with web pages "headless", that is, having the dom and js handled all in memory without any actual browser or window showing? I'd love to hear about personal experience / anecdotes, but just rattling off some names to research would be helpful too. Thanks!

1 0

lisa (muug.mb.ca) is down - change your bookmarks/contacts!
by Trevor Cordes 05 Sep '17

05 Sep '17

For everyone's information, our old server (still used for a couple of things) went down over the weekend. It has (shock, horror) bad capacitors. This is a reminder to make sure all of your MUUG web and email bookmarks have been changed to use the new server muug.ca, not the old muug.mb.ca. In particular, if you have email contacts/nicknames to write to the muug mailing lists, make sure they are set to the new (muug.ca) domain. We are working to make it so the old domain will still function, but it may not take effect for a day or two. In any event, one day we will decomission the old domain completely, so you might as well make your changes now. Thanks! And, as always, if you find something doesn't work as it should, email board(a)muug.ca

1 0

MUUG Meeting, Sept. 12, 7:30pm -- NTP, RCS Redux
by Gilbert E. Detillieux 05 Sep '17

05 Sep '17

*** Please note the updated meeting location below *** The Manitoba UNIX User Group (MUUG) will be holding its next monthly meeting on Tuesday, September 12. The meeting topic for this month is as follows: NTP Wyatt Zacharias will be presenting NTP, with a bit of theory and practical information on setting up NTPD. RTFM: RCS Redux Adam Thompson will be hosting this month's RTFM segment about Revision Control System. Like other version control systems, RCS automates the storing, retrieval, logging, identification, and merging of revisions. This primer will help you manage your frequently-revised text, source code, programs, documentation, graphics, papers, and form letters. The group usually holds its meetings at 7:30pm on the second Tuesday of every month from September to June. (There are no meetings in July and August.) Meetings are open to the general public; you don't have to be a MUUG member to attend. ******************************************************************* Please note our *NEW* meeting location for this month: *Room 1C16A* Centennial Hall, University of Winnipeg, entrance on Ellice Ave. between Spence St. and Balmoral St. Parking is available on the surrounding streets and in the lots on nearby streets. Look for signage once you're at the building, or ask a security guard. ******************************************************************* For more information about MUUG, and its monthly meetings, check out their web server: https://muug.ca/ Help us promote this month's meeting, by putting this poster up on your workplace bulletin board or other suitable public message board: https://muug.ca/meetings/MUUGmeeting.pdf -- Manitoba UNIX User Group E-mail: <gedetil(a)muug.ca> c/o Gilbert E. Detillieux Web: http://muug.ca/ University of Manitoba Phone: (204)474-8161 Winnipeg MB CANADA R3T 2N2 Fax: (204)474-7609

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

Roundtable September 2017