Just as an example, this is how it looks when it finds things not alphabetical, for AB.C at least.
https://travis-ci.org/acceptbitcoincash/acceptbitcoincash/jobs/335426778#L907

I hope you enjoy this simple functionality I added for our fork of your repository. Its helped us quickly address the fault instead of having to search through all the listings or reference the most recent set of changes for where it broke. Instead anyone submitting a PR can see this information and its easily made use of.

@kenman345 i took your idea and tweaked it a bit for the super long lists that twofactorauth has. i pushed a commit that uses a diff module for get contextual patch style diffs (showing exactly what needs to be changed) because otherwise some of the diffs were super long and hard to read

/usr/local/Cellar/ruby/2.5.0_2/bin/ruby ./verify.rb
<------------ ERROR ------------>
1. _data/email.yml not ordered by name. Correct order:
   phone: true
   doc: https://help.aol.com/articles/2-step-verification-stronger-than-your-password-alone
 - name: FastMail
   url: https://www.fastmail.com/
   img: fastmail.png
   tfa: true
   sms: true
   software: true
   hardware: true
   doc: https://www.fastmail.com/help/account/2fa.html
+- name: Freenet
+  url: https://www.freenet.de/
+  img: freenet.png
+  tfa: false
+  facebook: freenet
 - name: Gmail
   url: https://gmail.com
   img: gmail.png
   tfa: true
   sms: true
   phone: true
   software: true
   hardware: true
   doc: https://www.google.com/intl/en-US/landing/2step/features.html
-- name: Freenet
-  url: https://www.freenet.de/
-  img: freenet.png
-  tfa: false
-  facebook: freenet
 - name: GMX
   url: https://www.gmx.com/
   twitter: gmxmail
   img: gmx.png
   tfa: false
 - name: Hushmail
   url: https://www.hushmail.com
   img: hushmail.png
   tfa: true
   sms: true
rake aborted!

I like it! I've already brought it into a refactor bracnh we have on our repository so we also get the benefits

glad you brought this up! great idea! makes additions more approachable for novice users, which is great!

now i just need to find a simple Github App that allows travis to push errors programmatically into Github comments (so that users don't have to go to travis and scroll through logs to see them) ...

have you looked at https://www.travisbuddy.com/?

I havent tried it but have been curious

humm, i'll have to look, never seen

from what I've seen, it looks cool. He hasnt experimented with any interesting build strucutres but the dev is fairly responsive and welcoming of feedback and questions

i was really hoping to be able to send a customized message, not just steal the logs (since we have a bunch of other ruby cruft that gets output whenever there's an error)

Well, you only need a github token to use an endpoint for the github APIs to push something. So before it throws the error I guess it could collect the output and hit that with the global environment variable.

But yes, I'd be interested in something like that too. Just make sure it is wrapped in a check for it being a PR travis build before trying to push a comment to something

@stephengroat When I brought in Diffy on my repo it gave us this: Could not find gem 'diffy' in any of the gem sources listed in your Gemfile.

Mind taking a look?
acceptbitcoincash@6dfe9d9

I think it may be an issue with their local setup, will ask them to update their gems first

btw, are you on slack or discord or something we can chat more directly? would love to ask you some questions about the repository to help keep ours in shape.

@kenman345 for acceptbitcoincash/acceptbitcoincash@6dfe9d9 running bundle install && bundle exec rake, i get this for verify:

/usr/local/Cellar/ruby/2.5.0_2/bin/ruby ./verify.rb
Processing: _data/sections.yml
  1. img/food/cityLounge.png should not be larger than 2500 bytes. It is currently 3086 bytes.
  2. img/gambling/oddevenbets.png should not be larger than 2500 bytes. It is currently 2675 bytes.
  3. img/gambling/SportsBettingAG.png should not be larger than 2500 bytes. It is currently 2667 bytes.
  4. img/retail/scottmillsart.png should not be larger than 2500 bytes. It is currently 2663 bytes.
Processing: _data/adult-sections.yml
  No errors found
Processing: _data/donation-sections.yml
  No errors found
rake aborted!
Command failed with status (1): [/usr/local/Cellar/ruby/2.5.0_2/bin/ruby ./...]
/private/tmp/acceptbitcoincash/Rakefile:58:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.5.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:30:in `block in <main>'
/usr/local/bin/bundle:22:in `<main>'
Tasks: TOP => default => verify
(See full trace by running task with --trace)

are you sure that you're running bundle install && bundle exec rake and using the bundled gems?

Yea I'll work with them on this. Thanks!

I am thinking of revising that verify.rb script to check and report an error if the file is over 2500 kb but then subtract out a count of the imgs that are under 3000 kb by adding to a global variable. In our case we are okay with larger images as imgbot will compress so we dont want to make it a hard stop like it currently is.

btw, how did you get the travis build checks to indicate "required" ? We dont have that

@kenman345 required is in the github settings

most of those rubocop errors can be fixed with rubocop --auto-correct

Where are those settings?

take a look at how travis buddy is working for us:
acceptbitcoincash#761 (comment)

this will work out well

I recommended some changes in a PR here: #3083

I can add to that PR the travisbuddy call so that you can get the benefits of it without multiple PRs messing with .travis.yml file.

Alternatively, you can use this: https://probot.github.io/apps/ci-reporter/

Havent tried it myself but definitely considering it soon

Just wanted to mention another change I made to the AcceptBitcoinCash verify.rb script.

acceptbitcoincash@96fca6a

Essentially, when it is going to say the same URL is used in the same section, it gives the generic message but adds the output of the two listings.

Here is what the output looks like in a test:

We had a non-contributor make a PR and forget to make that same change and I had to go in to correct it. It happens, but they were completely thrown off by why the build failed and needed to wait for intervention. This change would add in better understanding that would be helpful in allowing users to understand the mistake and correct it themselves without additional help.

Wanted to also point out I ran the metavalidator against the website_schema.yml file on AB.C and it failed due to improper URL pattern matching. I made the proper change, figured you might like to take a peak: https://github.com/acceptbitcoincash/acceptbitcoincash/pull/807/files

ding ding, just pointing out this still hasnt been approved. Anything I can do to help? I know it was mentioned that we need a way to show the user when Travis fails and I've explained an inclusion we can make, just let me know whats happening, thanks

Hey, so I run off windows and in windows, I not only needed diffy but also rspec as the easiest way to get a 'Diff' recognized by windows. might want to add that in.

or even just 'diff-lcs' by itself helps.

whats going on with this?

I think for Windows users its nice if we add in gem 'diff-lcs' to ensure they dont have issues getting the error message when diffy outputs the correct order.

And Travis-Buddy works very well for commenting on PRs with the results of your test to help indicate the error itself.

EDIT: Also, I have not ported over the change that adds in the indication of the actual values that are invalid when the website schema indicates that a name or URL is not unique between listings. I can open a new PR for that after this one is merged since they're somewhat separate functionality.

I was seeing how long this PR is taking and wanted to bring further enhancements to it but was unsure if its worth waiting and making a separate PR or adding to this one.

First is I really think ensuring we have compatibility for Windows users is important so adding a gem dependency would make sure the added 'diffy' gem always works properly and is a great experience for all. I mentioned the gem diff-lcs in my comment above.

Also, I was thinking that it would be the right call to switch to using the SafeYaml library for opening the yaml files. If someone made a PR and it had dangerous code, it would be able to run and cause harm to other contributoirs that run this locally. If you simply swap out the YAML.load_file( calls to SafeYaml.load_file then you get it to safely open the files and not accidentally execute code if it ever were to get into your repository.

Shall I go ahead with that change/changes?

Also, I was thinking that it would be the right call to switch to using the SafeYaml library for opening the yaml files.

Not so sure on that. SafeYaml is 4 years old and has no test coverage metrics. An alternative would be Psych's safe_load function that's in Ruby's standard modules. Shouldn't really make a difference, if you want to try YAML::safe_load I don't see any compelling reason not to.

Gives the user contextual information about how to alphabetize the file. With different characters (spaces, symbols, etc.) I think it provides quite a bit of information.

I don't see how a slightly longer output is that bad. We can always tweak the context parameter to give more or less lines if needed

If we know how it should be sorted, is it feasible to automatically update PR with correct sorting?
Or perhaps output it as a PR comment rather than user digging for travis errors?

#3067 (comment) I think this is the first step, hopefully bot development will help us achieve something in the future

@stephengroat just wanted to add, that Safe YAML is supported officially by Github, so it may be 4 years old but its still relevant https://pages.github.com/versions/

Just because it's included by the github-pages gem doesn't mean it's officially support by GitHub. No issues have been resolved in years and there's been no PRs merged in a while as well. Interesting that GitHub chose to include it, but doesn't mean that it's supported

Whats gotta happen for this to move along? Am I to remove the SafeYAML in place of plain old YAML thats not safe? SafeYAML doesnt need to be actively worked on and I'd happily update the gem myself if it were needed. The gem itself is already downloaded when people run locally thanks to Github pages containing it. It also doesnt need to be updated as its a wrapper for the YAML ruby component to ensure we use the right safe options. The options themselves havent changed recently and dont look to be changing much in the future, especially in regards to opening a new file.