more seed data (#2356) #2361

omenking · 2019-04-10T13:46:44Z

What type of PR is this? (check all applicable)

Refactor
Feature
Bug Fix
Documentation Update

Description

I add more seed data, I also add a new user which has a populated dashboard which you can impersonate only in development via /quickin

Running seed takes much more time now. We can dial back the number of objects created and let the developer decide if they need to create more of X for their specific development use cases to mitigate this issue.

Related Tickets & Documents

This is the original ticket where I document the journey:
#2356

I needed this to complete this ticket:
#258

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

Added to documentation?

docs.dev.to
readme
no documentation needed

[optional] What gif best describes this PR or how it makes you feel?

omenking · 2019-04-15T17:47:46Z

brought this ticket back up to date with current master, awaiting its merge so I may complete:
#258

app/controllers/users_controller.rb

rhymes · 2019-04-15T19:00:59Z

db/seeds.rb


-3.times do
+40.times do


What about using a multiplicator as a parameter (in the case of db:seed it'd probably be an env variable)?

Let's say the default seeds keeps things as they are and one can say "generate 2 times the default amount of data" or 10 times that.

I increased the number enough for the purpose of pagination, and I think we should ensure we have enough data by default so when people go to Q&A the app has what they need without having to look or fiddle with settings.

In honest, this is such a small increase in seed data where I have web-apps which load 10K records within a minute or less.

The seed data is slow in the repo because of architectural choices such as using acts_as_taggable. I can certainly suggest in future tickets to improve these performance issues so we can have much more seed data.

rhymes · 2019-04-15T19:02:06Z

db/seeds.rb

+
+100.times do |i|
+  name = Faker::Name.unique.name
+  puts "username #{Faker::Internet.username(name)}"


I wouldn't display 100 (or a 1000 in case of the user's multiplication factor) user names on the console, what's to gain?

If you are testing out pagination, you would.

rhymes · 2019-04-15T19:06:27Z

db/seeds.rb

@@ -213,6 +217,67 @@
 )
 ##############################################################################

+Rails.logger.info "9/10 Devvy Ruxpin"


What if, instead of having "Devvy Ruxpin" as a seed and resort to the hack of having the sign in as a user, all the following code would moved to a regular rake task that can only be run in development mode and that, by passing the username or user id, would do all the "followings" and create all the articles?

This way you don't need any hack. If you want a regular seeded DB you just run the normal seed, if you want more data you increase it with the multiplication factor and if you need to follow everything you just run a rake task meant for dev purposes.

An alternative to the quickin would be to use a gem such as Pretender which would only work within the admin panel.

I actually don't know if an admin panel exists or how to access it in this codebase which is why I had not suggested this previously.

I don't understand the objections to increasing the amount of data as currently the seed data is insufficent for general Q&A.

It's not an objection, I'm saying that if you make it parameterized the user can decide how much seed data they want and it's future proof :) You can still raise the default multiplier to have a better seed data default.

I'm not familiar with Pretender.

There's an admin, it's under /admin. There are also internal pages which I think it's a "side admin", see the routes under /internal

Why not just give them the right amount of seed data to test the app without configuration? In order to Q&A various pagination you need more seed data.

Sounds like neither of you object to making the amount of data configurable, just what the default quantity of data should be. @omenking wants to increase the default quantity. @rhymes, are you ok with increasing that amount, or do you want the current quantity to be the default? (I agree with @omenking that the amount of seed data should be sufficient to exercise paging.)

@keithrbennett I still think it should be a configurable param, we can also tweak the default later

Zhao-Andy · 2019-04-15T19:13:33Z

config/routes.rb

@@ -280,6 +280,9 @@
    get "/rails/mailers/*path" => "rails/mailers#preview"
  end

+  get '/quickin'     => 'users#quickin'


These two routes should also be wrapped in an if Rails.env.development? statement.

This is a good point. I had never considered whether Rails.env could be used in routes but its ruby so certainly should add.

omenking · 2019-04-15T20:17:14Z

Now that I'm aware there is an admin panel, I think I should have that default user I have in this seed data have admin access as soon as I determine how it can be applied. There is no admin column on the users table, the roles table doesn't appear to have admin. I'm looking up devise methods such as authorize buts it's not helping me yet determine how to set a user as an admin.

Also since diff-coverage is complaining about lack of test code I'll see if I can make it happy.

rhymes · 2019-04-15T20:31:22Z

@omenking the roles are stored in the roles table by the gem rolify AFAIK. You can call user.add_role(). I don't know much more about that, there's a role :admin and a role :super_admin and others. You can find them in the role model:

https://github.com/thepracticaldev/dev.to/blob/53381b9c0d3f4e9ed1d42d62e2988b93124f7b1f/app/models/role.rb#L14-L32

abraham · 2019-04-15T20:35:49Z

If you switch to using FactoryBot you can do FactoryBot.create(:user, :super_admin) and it will return a created user with that role.

omenking · 2019-04-15T21:00:14Z

@abraham It's not recommended by the gem creators: FactoryGirl for Seed Data?. It's an old article but I think it still applies.

Zhao-Andy · 2019-04-15T21:02:25Z

@omenking authorize is how we handle authorization via the Pundit gem (as opposed to authentication, which is handled by Devise). Related code and files are found in /app/policies.

omenking · 2019-04-17T01:19:27Z

its a busy week but I'll find time to get these improvements rolled in.

benhalpern · 2019-04-19T16:40:49Z

Running seed takes much more time now.

What if we made the seed file work with activerecord-import to speed everything up?

omenking · 2019-04-19T16:46:58Z

Will investigate activerecord-import and report back

omenking · 2019-04-23T14:59:00Z

@benhalpern I looked at activerecord-import and it appears to be taking the approach I would use to speed up seed data. In execution, I think it may not work because it will either skip all the callbacks by act_as_taggable or they will still run and that's where the bottleneck for seed data is.

The gem is quite hefty and I am adverse into increasing the Gemfile further. I'm discouraged to write a simple implementation of what the gem does because the peer review will just delay this PR further.

In honest, I think I'm going to abandon this ticket and PR because I only have a few hours a week to work on DEV.to and I didn't want to spend more than 30min here, it was simply to add more seed data for QA of other features.

omenking · 2019-04-25T19:57:05Z

@Zhao-Andy is there a way to ignore coverage on files such as tags.txt and seeds.rb
I won't be able to get this to pass diff-coverage otherwise.

Zhao-Andy · 2019-04-25T20:00:54Z

@maestromac would know better than I would, but I think it's okay if the diff coverage doesn't pass for this case.

rhymes · 2019-09-30T16:09:32Z

Hi @omenking, we're going to close this for the following two reasons:

more recently we had additional discussion around the idea of the quick login around this PR Add a /quickin route for development #3893 but we'll have more to talk about, though we were leaning on the idea of having an external gem by DEV for that feature
regarding the seed data we feel it should probably be paramaterized (either via rake task arguments or environment variables or other means)

Sorry if we let this PR slip for so long.

I'm going to open an issue about the seed data so that if we wanto we can discuss around that before coming to an agreeable implementation

more seed data (forem#2356)

8980dbe

pr-triage bot added the PR: unreviewed bot applied label for PR's with no review label Apr 10, 2019

Merge branch 'master' into feature/more-seed-data

66f4b1b

rhymes reviewed Apr 15, 2019

View reviewed changes

Zhao-Andy reviewed Apr 15, 2019

View reviewed changes

weekly-digest bot mentioned this pull request Apr 29, 2019

Weekly Digest (22 April, 2019 - 29 April, 2019) #2598

Closed

Merge branch 'master' into feature/more-seed-data

fd422ef

weekly-digest bot mentioned this pull request May 27, 2019

Weekly Digest (20 May, 2019 - 27 May, 2019) #2967

Closed

rhymes mentioned this pull request Sep 2, 2019

Add a /quickin route for development #3893

Closed

7 tasks

rhymes closed this Sep 30, 2019

rhymes mentioned this pull request Sep 30, 2019

Increase seed data for development purposes #4149

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

more seed data (#2356) #2361

more seed data (#2356) #2361

omenking commented Apr 10, 2019

omenking commented Apr 15, 2019

rhymes Apr 15, 2019

omenking Apr 15, 2019 •

edited

Loading

rhymes Apr 15, 2019

omenking Apr 15, 2019

rhymes Apr 15, 2019

omenking Apr 15, 2019

rhymes Apr 15, 2019

omenking Apr 15, 2019

keithrbennett Dec 24, 2019

rhymes Dec 24, 2019

Zhao-Andy Apr 15, 2019

omenking Apr 15, 2019

omenking commented Apr 15, 2019

rhymes commented Apr 15, 2019

abraham commented Apr 15, 2019

omenking commented Apr 15, 2019

Zhao-Andy commented Apr 15, 2019

omenking commented Apr 17, 2019

benhalpern commented Apr 19, 2019

omenking commented Apr 19, 2019

omenking commented Apr 23, 2019 •

edited

Loading

omenking commented Apr 25, 2019

Zhao-Andy commented Apr 25, 2019

rhymes commented Sep 30, 2019


		3.times do
		40.times do

more seed data (#2356) #2361

more seed data (#2356) #2361

Conversation

omenking commented Apr 10, 2019

What type of PR is this? (check all applicable)

Description

Related Tickets & Documents

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

Added to documentation?

[optional] What gif best describes this PR or how it makes you feel?

omenking commented Apr 15, 2019

Choose a reason for hiding this comment

omenking Apr 15, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

omenking commented Apr 15, 2019

rhymes commented Apr 15, 2019

abraham commented Apr 15, 2019

omenking commented Apr 15, 2019

Zhao-Andy commented Apr 15, 2019

omenking commented Apr 17, 2019

benhalpern commented Apr 19, 2019

omenking commented Apr 19, 2019

omenking commented Apr 23, 2019 • edited Loading

omenking commented Apr 25, 2019

Zhao-Andy commented Apr 25, 2019

rhymes commented Sep 30, 2019

omenking Apr 15, 2019 •

edited

Loading

omenking commented Apr 23, 2019 •

edited

Loading