Skip to content

[Feature Request] Ability to fetch a single page of items in a list #406

Open

Description

@jeanettehead

Would be nice to be able to fetch only a page of repositories, repo commits, and other lists instead of fetching the entire thing every time. Currently they call into _requestAllPages which automatically fetches everything from github, which can take awhile for long lists.

@mtscout6 suggested on gitter that a generator function would be a good way to accomplish this.

If there's a good direction to go in, I can probably work on this.

Activity

  1. mtscout6 commented on Dec 2, 2016

    @mtscout6
    Member

    So this gets interesting because there's a few ways to do this. First is to be transparent about paging. So, something like:

    for (let page of request()) {
      for (let item of page.items) { ... }
    }
    

    Can't say that I'm a huge fan of that approach. Though if you move to using the newer for await support from Babel as proposed with the async generator transform and outlined as a tc39 proposal

    Then you could do something like this:

    for await (let item of request()) {
      ...
    }
    

    Though the ES5 equivalent gets really messy:

    function forEach(ai, fn) {
      return ai.next().then(function (r) {
        if (!r.done) {
          fn(r);
          return forEach(ai, fn);
        }
      });
    }
    
    var item = 0;
    forEach(request(), function(val) { item += val.value })
      .then(function () {
        console.log(item);
      });
    

    Essentially downstream users that use Babel would be the ones to win here, those not Using Babel would need to wait a few years for the Spec changes to replicate to the different JavaScript interpreters. If we did the async generator approach then we should callout in the docs that Babel is recommended and that we wouldn't support users not using Babel.

    @clayreimann Thoughts?

  2. jeanettehead commented on Dec 3, 2016

    @jeanettehead
    ContributorAuthor

    I'm down for that. A (stupidly simple) alternative would be to allow the users to send a page option through and fetch just that page. As it stands, sending a page option causes an infinite loop of requests, which seems like a bug.

  3. jeanettehead commented on Dec 15, 2016

    @jeanettehead
    ContributorAuthor

    @mtscout6 I'd still like to make this happen. I'm down for either approach - do you have any thoughts?

  4. mtscout6 commented on Dec 15, 2016

    @mtscout6
    Member

    The page option is a valid option, and much simpler too. Go ahead and submit a PR for it.

  5. jeanettehead commented on Dec 20, 2016

    @jeanettehead
    ContributorAuthor

    Opened a pull request: #409

  6. jeanettehead commented on Jan 20, 2017

    @jeanettehead
    ContributorAuthor

    This would be all set, but when I npm install https://github.com/github-tools/github.git#3.1.0, it's unable to be imported into my project because it can't find the dist/components/GitHub.js specified as the main entry point in the package.json. Is this fixable or can a new release get published to npm?

  7. jeanettehead commented on Jan 27, 2017

    @jeanettehead
    ContributorAuthor

    @mtscout6 @clayreimann any update on my above question?

  8. mtscout6 commented on Jan 27, 2017

    @mtscout6
    Member

    That dist folder is generated during the CI. With the CI failing and not publishing to npm that's your problem. I have not had time to dig into the cause of the CI failure, and I do not have permissions to push directly to npm bypassing CI. I wish I could say that I had the time to look at it, but I don't for some time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions